-- Logs begin at Mon 2022-06-27 20:26:08 UTC, end at Mon 2022-06-27 21:34:08 UTC. -- Jun 27 21:01:33.176298 dv-cinder-847732-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 27 21:01:33.187348 dv-cinder-847732-1-iscsi devstack@keystone.service[80378]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 27 21:01:33.189440 dv-cinder-847732-1-iscsi devstack@keystone.service[80378]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jun 27 21:01:33 2022] *** Jun 27 21:01:33.189440 dv-cinder-847732-1-iscsi devstack@keystone.service[80378]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 27 21:01:33.189440 dv-cinder-847732-1-iscsi devstack@keystone.service[80378]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 27 21:01:33.189440 dv-cinder-847732-1-iscsi devstack@keystone.service[80378]: nodename: dv-cinder-847732-1-iscsi Jun 27 21:01:33.189440 dv-cinder-847732-1-iscsi devstack@keystone.service[80378]: machine: x86_64 Jun 27 21:01:33.189440 dv-cinder-847732-1-iscsi devstack@keystone.service[80378]: clock source: unix Jun 27 21:01:33.189440 dv-cinder-847732-1-iscsi devstack@keystone.service[80378]: pcre jit disabled Jun 27 21:01:33.189440 dv-cinder-847732-1-iscsi devstack@keystone.service[80378]: detected number of CPU cores: 4 Jun 27 21:01:33.189440 dv-cinder-847732-1-iscsi devstack@keystone.service[80378]: current working directory: / Jun 27 21:01:33.189440 dv-cinder-847732-1-iscsi devstack@keystone.service[80378]: detected binary path: /usr/bin/uwsgi-core Jun 27 21:01:33.189440 dv-cinder-847732-1-iscsi devstack@keystone.service[80378]: your processes number limit is 47680 Jun 27 21:01:33.189440 dv-cinder-847732-1-iscsi devstack@keystone.service[80378]: your memory page size is 4096 bytes Jun 27 21:01:33.189440 dv-cinder-847732-1-iscsi devstack@keystone.service[80378]: detected max file descriptor number: 2048 Jun 27 21:01:33.189440 dv-cinder-847732-1-iscsi devstack@keystone.service[80378]: lock engine: pthread robust mutexes Jun 27 21:01:33.189440 dv-cinder-847732-1-iscsi devstack@keystone.service[80378]: thunder lock: enabled Jun 27 21:01:33.190476 dv-cinder-847732-1-iscsi devstack@keystone.service[80378]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 27 21:01:33.190476 dv-cinder-847732-1-iscsi devstack@keystone.service[80378]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 27 21:01:33.239710 dv-cinder-847732-1-iscsi devstack@keystone.service[80378]: Python main interpreter initialized at 0x561e6b1579c0 Jun 27 21:01:33.239710 dv-cinder-847732-1-iscsi devstack@keystone.service[80378]: python threads support enabled Jun 27 21:01:33.239710 dv-cinder-847732-1-iscsi devstack@keystone.service[80378]: your server socket listen backlog is limited to 100 connections Jun 27 21:01:33.239710 dv-cinder-847732-1-iscsi devstack@keystone.service[80378]: your mercy for graceful operations on workers is 90 seconds Jun 27 21:01:33.240129 dv-cinder-847732-1-iscsi devstack@keystone.service[80378]: mapped 403077 bytes (393 KB) for 2 cores Jun 27 21:01:33.240129 dv-cinder-847732-1-iscsi devstack@keystone.service[80378]: *** Operational MODE: preforking *** Jun 27 21:01:33.240441 dv-cinder-847732-1-iscsi devstack@keystone.service[80378]: *** uWSGI is running in multiple interpreter mode *** Jun 27 21:01:33.240441 dv-cinder-847732-1-iscsi devstack@keystone.service[80378]: spawned uWSGI master process (pid: 80378) Jun 27 21:01:33.240628 dv-cinder-847732-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 27 21:01:33.245108 dv-cinder-847732-1-iscsi devstack@keystone.service[80378]: spawned uWSGI worker 1 (pid: 80388, cores: 1) Jun 27 21:01:33.245108 dv-cinder-847732-1-iscsi devstack@keystone.service[80378]: spawned uWSGI worker 2 (pid: 80389, cores: 1) Jun 27 21:01:33.245108 dv-cinder-847732-1-iscsi devstack@keystone.service[80378]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 27 21:01:38.694779 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 27 21:01:38.694779 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: from cryptography.utils import int_from_bytes Jun 27 21:01:38.695641 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 27 21:01:38.695641 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: from cryptography.utils import int_from_bytes Jun 27 21:01:38.696047 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 27 21:01:38.696047 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: from cryptography.utils import int_from_bytes Jun 27 21:01:38.696146 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 27 21:01:38.696146 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: from cryptography.utils import int_from_bytes Jun 27 21:01:38.744415 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 27 21:01:38.744618 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 27 21:01:38.744696 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 27 21:01:38.744793 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 27 21:01:38.744863 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] command line args: [] {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 27 21:01:38.744961 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] command line args: [] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 27 21:01:38.745036 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 27 21:01:38.745103 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 27 21:01:38.745262 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ================================================================================ {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 27 21:01:38.745349 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ================================================================================ {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 27 21:01:38.745467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] admin_token = **** {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.745558 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] admin_token = **** {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.745753 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] config_dir = [] {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.745889 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] config_dir = [] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.745889 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.746025 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.746085 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] config_source = [] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.746156 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] config_source = [] {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.746216 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] debug = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.746275 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] debug = True {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.746432 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.746675 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.746766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.746826 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.746954 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.747065 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.747184 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.747280 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.747426 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.747544 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.747654 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] list_limit = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.747748 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] list_limit = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.747850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] log_config_append = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.747945 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] log_config_append = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.748051 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.748147 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.748247 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] log_dir = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.748343 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] log_dir = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.748448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] log_file = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.748551 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] log_file = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.748644 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.748744 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.748850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.748996 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.749086 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.749196 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.749312 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.749432 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.749515 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.749583 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.749707 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.749791 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.749854 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.749966 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.750084 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.750168 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.750283 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.750358 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.750461 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.750554 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.750716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.750854 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.750989 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.751087 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.751165 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.751292 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.751368 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.751489 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.751669 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.751794 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.752172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.752297 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.752393 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] publish_errors = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.752469 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] publish_errors = False {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.752597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.752694 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.752769 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.752893 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.753007 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.753133 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.753209 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.753344 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.753429 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.753504 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.753620 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] standard_threads = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.753701 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] standard_threads = False {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.753917 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.754000 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.754078 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.754159 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.754237 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.754314 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.754394 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] use_journal = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.754505 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] use_journal = False {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.754590 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] use_json = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.754664 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] use_json = False {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.754734 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] use_stderr = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.754813 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] use_stderr = False {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.754886 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] use_syslog = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.754956 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] use_syslog = False {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.755106 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.755231 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 27 21:01:38.755310 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.755397 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.755579 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.755705 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.755818 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.755904 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.756077 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.756216 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.756305 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.756379 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.756549 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.756684 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.756773 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.756858 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.757031 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.757164 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.757244 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.757317 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.757508 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.757652 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.757747 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.757822 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.757977 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.758103 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.758193 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.758290 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.758447 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.758577 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.758657 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.758730 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.758914 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.759130 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.759220 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.759295 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.759424 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.759552 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.759773 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.759905 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.760027 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.760146 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.760259 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.760371 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.760482 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.760605 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.760725 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.760840 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.760990 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.761105 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.761217 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.761343 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.761464 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.761577 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.761688 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.761800 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.761911 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.762032 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.762145 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.762263 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.762382 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.762495 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.762615 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.762728 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.762839 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.762950 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.763062 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.763174 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.763297 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.763410 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.763527 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.763639 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.763750 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.763861 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.763980 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.764093 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.764204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.764315 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.764426 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.764536 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.764729 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.764845 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.764971 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.765084 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.765195 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.765306 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.765430 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.765549 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.765668 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.765780 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.765892 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.766009 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.766130 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.766249 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.766363 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.766474 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.766591 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.766709 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.766834 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.766946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.767063 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.767168 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.767264 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.767360 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.767464 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.767561 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.767656 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.767751 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.767852 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.767993 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.768107 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.768206 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.768313 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.768408 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.768503 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.768599 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.768713 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.768811 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.768942 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.769068 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.769174 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.769284 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.769410 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.769532 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.769637 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.769747 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.769859 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.769984 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.770107 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.770226 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.770352 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.770466 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.770584 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.770706 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.770819 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.770935 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.771057 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.771177 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.771288 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.771402 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.771515 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.771626 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.771739 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.771857 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.771971 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.772083 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.772195 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.772309 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.772441 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.772556 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.772667 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.772779 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.772882 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.773062 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.773178 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.773291 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.773410 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.773525 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.773637 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.773758 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.773872 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.773985 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.774104 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.774235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.774336 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.774456 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.774571 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.774683 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.774795 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.774906 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.775025 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.775149 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.775279 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.775391 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.775504 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.775617 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.775729 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.775921 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.776045 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.776159 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.776270 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.776382 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.776494 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.776494 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.776724 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.776849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.776984 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.777109 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.777224 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.777342 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.777456 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.777580 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.777694 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.777807 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.777923 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.778036 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.778147 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.778269 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.778382 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.778503 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.778616 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.778728 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.778827 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.778932 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.779030 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.779125 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.779203 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.779264 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.779323 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.779391 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] auth.external = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.779453 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.779512 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.779571 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] auth.external = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.779630 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.779698 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.779944 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.780084 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.780210 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] auth.password = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.780326 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.780439 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] auth.token = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.780552 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] auth.password = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.780665 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.780787 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] auth.token = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.780903 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.781084 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.781200 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.781315 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.781437 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.781560 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.781675 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.781789 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.781904 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.782009 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.782108 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.782214 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.782322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] credential.caching = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.782420 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.782532 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.782645 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] credential.caching = True {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.782766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.782889 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.783005 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.783119 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.783232 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.783351 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.783465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.783585 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.783700 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.783819 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.783933 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.784054 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.784168 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.784289 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.784403 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.784517 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.784631 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.784745 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.784858 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.784996 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.785114 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.785228 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.785342 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.785455 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.785568 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.785689 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.785803 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.785929 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.786043 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] federation.caching = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.786151 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.786262 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.786383 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] federation.caching = True {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.786498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.786610 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.786723 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.786836 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.786949 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.787073 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.787188 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.787303 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.787416 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.787529 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.787643 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.787765 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.787880 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.788006 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.788119 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.788234 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.788347 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.788468 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.788583 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.788704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.788817 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] identity.caching = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.788981 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.789109 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.789109 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.789415 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] identity.caching = True {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.789529 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.789642 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.789754 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.789875 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.789990 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.790103 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.790216 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.790330 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.790442 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.790567 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.790682 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.790809 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.790957 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.791085 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.791206 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.791331 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.791447 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.791560 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.791673 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.791786 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.791899 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.792021 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.792135 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.792248 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.792366 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.792488 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.792608 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.792731 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.792847 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.792978 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.793094 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.793207 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.793322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.793448 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.793565 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.793678 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.793791 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.793904 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.794018 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.794140 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.794255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.794368 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.794483 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.794595 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.794709 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.794830 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.794945 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.795059 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.795178 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.795293 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.795406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.795529 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.795659 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.795817 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.795933 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.796047 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.796159 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.796281 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.796395 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.796508 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.796621 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.796725 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.796836 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.796975 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.797092 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.797205 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.797318 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.797431 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.797543 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.797669 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.797784 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.797896 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.798014 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.798127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.798240 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.798362 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.798476 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.798590 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.798704 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.798817 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.798937 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.799060 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.799174 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.799287 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.799399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.799512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.799625 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.799747 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.799861 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.799974 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.800087 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.800200 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.800313 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.800435 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.800550 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.800663 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.800777 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.user = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.800890 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.801057 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.801184 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.801299 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.801412 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.user = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.801524 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.801637 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.801758 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.801877 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.801990 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.802103 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.802216 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.802328 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.802456 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.802573 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.802686 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.802799 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.802914 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.803027 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.803148 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.803263 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.803377 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.803490 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.803603 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.803715 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.803840 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.803956 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.804068 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.804181 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.804294 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.804407 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.804528 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.804642 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.804754 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.804867 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.805003 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.805118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.805243 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.805358 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.805472 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.805585 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.805698 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.805698 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.805918 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.806034 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.806146 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.806146 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.806358 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.806483 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.806606 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.806728 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.806840 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.806943 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.807041 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.807139 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.807249 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.807364 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.807477 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.807477 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.807689 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.807803 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.807928 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.808043 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.808156 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.808269 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.808382 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.808495 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.808615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.808730 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.808842 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.809002 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.809118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.809233 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] resource.caching = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.809357 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.809472 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.809585 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.809697 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.809810 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.809922 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.810043 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] resource.caching = True {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.810158 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.810272 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.810386 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.810498 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.810611 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.810732 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.810847 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.810963 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.811076 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.811188 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.811301 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.811493 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.811609 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] role.caching = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.811723 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.811837 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] role.driver = sql {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.811950 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.812064 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.812185 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.812301 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.812413 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] role.caching = True {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.812526 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.812639 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] role.driver = sql {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.812754 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.812875 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.813002 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.813116 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.813223 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.813334 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.813460 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.813576 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.813689 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.813802 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.813915 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.814027 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.814149 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.814263 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.814377 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.814490 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.814603 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.814716 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.814837 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.814952 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.815065 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.815445 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.815445 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.815675 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.815796 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.815912 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.816025 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.816139 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.816139 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.816350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.816472 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.816587 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.816700 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.816813 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.816981 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.817098 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.817226 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.817341 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.817454 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.817568 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.817682 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.817797 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.817922 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.818037 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.818151 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.818264 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.818376 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.818489 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.818611 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.818725 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.818837 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.818950 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.819062 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.819183 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.819298 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.819411 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.819524 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.819638 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.819751 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.819873 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.819988 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] token.caching = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.820101 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.820214 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.820327 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.820440 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.820566 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.820681 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.820794 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] token.caching = True {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.820907 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.821036 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.821150 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.821276 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.821391 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.821505 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.821617 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.821731 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.821846 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.821967 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.822210 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.822338 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.822453 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.822566 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.822680 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.822804 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.822920 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.823035 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.823147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.823261 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.823373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.823495 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.823610 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.823723 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.823838 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.823951 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.824064 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.824186 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.824186 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.824401 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] database.connection = **** {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.824524 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.824655 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.824655 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.824870 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.825043 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.825171 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] database.connection = **** {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.825296 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.825420 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.825534 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.825647 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.825759 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.825872 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.825995 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.826111 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.826224 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.826337 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.826451 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.826564 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.826685 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.826799 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.826913 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.827027 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.827140 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.827252 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.827379 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.827494 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.827607 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.827719 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.827832 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.827948 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.828072 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.828186 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.828298 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.828410 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.828522 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.828634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.828755 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80388) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 27 21:01:38.828871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.828871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.828871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 27 21:01:38.828871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 27 21:01:38.859351 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:38.859596 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:38.859736 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:38.860022 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:38.860265 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:38.860490 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:38.868502 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:38.869413 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:38.897433 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:38.897636 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:38.897781 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:38.898022 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.044343 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.045247 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.093509 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.093761 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.093857 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.093959 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.159136 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.159338 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.159430 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.159570 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.159680 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.159801 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.159912 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.160012 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.160134 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.160218 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.160281 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.160350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.186378 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.186611 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.186726 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.186819 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.186991 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.187117 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.187202 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.187342 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.187426 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.187503 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.187634 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.187722 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.187844 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.188064 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.199169 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.199662 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.212125 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.212359 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.212601 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.212821 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.213039 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.213171 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.213328 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.213422 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.213499 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.213574 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.213647 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.213719 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.213795 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.214041 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80388) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 27 21:01:39.231589 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.231786 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 27 21:01:39.232038 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.232146 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.232226 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 27 21:01:39.232301 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 27 21:01:39.232573 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.232664 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.232739 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 27 21:01:39.232822 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.233033 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.233133 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.233295 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.233386 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.233548 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.233801 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.244401 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.244675 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.244773 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.244945 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.245053 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.245241 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.245330 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.245518 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.253668 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80389) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:01:39.253858 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:01:39.256949 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80389) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:01:39.257161 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:01:39.257313 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.257438 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.257539 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.257743 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.257868 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.257994 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.258085 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.258299 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.258437 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.258519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.258594 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.258803 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.258928 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.259024 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.259100 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.259840 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.260335 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.260335 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.260520 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.260520 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.286405 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.286548 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.287969 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80389) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:01:39.288122 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:01:39.288206 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.288336 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.293741 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:01:39.293869 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80389) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:01:39.294104 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.294218 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.294384 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.294473 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.301742 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80389) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:01:39.301879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:01:39.302023 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.302113 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.306544 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80389) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:01:39.306660 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:01:39.306798 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.306900 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.307102 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.307192 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.307396 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.307486 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.307670 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.307770 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.307942 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.308030 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.308208 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.308296 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.308471 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.308558 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.327284 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.327432 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.327568 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.327669 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.332159 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:01:39.332287 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80389) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:01:39.335822 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.335946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.336223 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.336322 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.341353 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80389) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:01:39.341465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:01:39.344949 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:01:39.345054 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80389) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:01:39.348621 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.348762 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.348886 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.349048 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.349197 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.349308 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.349502 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.349629 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.349789 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.349915 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.350051 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.350142 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.369736 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80389) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:01:39.370013 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:01:39.370140 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.370301 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.370382 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.370528 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.370653 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.370788 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.378507 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.378716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.380211 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.380350 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.380487 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.380693 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.383698 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80389) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:01:39.383823 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:01:39.383957 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.384128 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.384253 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.384384 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.384475 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.384804 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.384958 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.385103 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.398242 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80389) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:01:39.398396 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:01:39.398551 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.398634 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.398753 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.398883 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.399071 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.399197 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.400878 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.401040 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.401174 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.401268 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.401493 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.401620 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.419379 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:01:39.419541 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80389) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:01:39.422667 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:01:39.422776 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80389) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:01:39.425983 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.426086 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.427695 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.427785 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.429338 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:01:39.429477 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80389) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:01:39.429601 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.429720 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.429892 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.430024 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.437356 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.437498 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.438913 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:01:39.439064 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80389) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:01:39.442177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.442337 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.442431 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.442630 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.442838 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.443038 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.443126 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.443326 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.452332 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:01:39.452618 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.452725 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80389) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:01:39.452893 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.453045 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.453216 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.460760 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80388) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:01:39.461060 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.461187 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80389) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 27 21:01:39.461349 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.461479 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.461563 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.461640 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.461863 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.461992 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.462193 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.462320 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.462500 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.462639 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.462771 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.462865 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.463071 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.463266 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.463354 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.463432 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.463644 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.463771 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.463872 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.464031 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.464160 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.464274 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.464432 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80388) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.464562 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.464806 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 27 21:01:39.502685 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x561e6b1579c0 pid: 80388 (default app) Jun 27 21:01:39.503506 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x561e6b1579c0 pid: 80389 (default app) Jun 27 21:01:39.506235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415bb67-8b15-4962-ba75-2fec74637a0b None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:39.506524 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415bb67-8b15-4962-ba75-2fec74637a0b None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:39.506800 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415bb67-8b15-4962-ba75-2fec74637a0b None None] PATH_INFO: `/v3` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:39.511453 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1/1] 10.222.0.13 () {50 vars in 791 bytes} [Mon Jun 27 21:01:39 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:01:47.325988 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560357ce-fabe-46d8-bca5-10ed8b2353ee None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:47.326560 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560357ce-fabe-46d8-bca5-10ed8b2353ee None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:47.326560 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560357ce-fabe-46d8-bca5-10ed8b2353ee None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:47.330865 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1/2] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:01:47 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:01:47.336635 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717eac59-6132-4898-a5d5-758774d5b465 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:47.337107 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717eac59-6132-4898-a5d5-758774d5b465 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:47.337199 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717eac59-6132-4898-a5d5-758774d5b465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:47.405798 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: INFO dbcounter [None req-717eac59-6132-4898-a5d5-758774d5b465 None None] Registered counter for database keystone Jun 27 21:01:47.427619 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG oslo_db.sqlalchemy.engines [None req-717eac59-6132-4898-a5d5-758774d5b465 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80388) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 27 21:01:47.435535 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writer thread running {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 27 21:01:47.625634 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG passlib.handlers.bcrypt [None req-717eac59-6132-4898-a5d5-758774d5b465 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80388) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 27 21:01:47.625980 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG passlib.handlers.bcrypt [None req-717eac59-6132-4898-a5d5-758774d5b465 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80388) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 27 21:01:47.681561 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-717eac59-6132-4898-a5d5-758774d5b465 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:01:47.683248 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-717eac59-6132-4898-a5d5-758774d5b465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:01:47.819144 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 2/3] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:01:47 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 483 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 27 21:01:47.952046 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560357ce-fabe-46d8-bca5-10ed8b2353ee None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:01:48.023157 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: INFO dbcounter [None req-560357ce-fabe-46d8-bca5-10ed8b2353ee None None] Registered counter for database keystone Jun 27 21:01:48.043246 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG oslo_db.sqlalchemy.engines [None req-560357ce-fabe-46d8-bca5-10ed8b2353ee None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80389) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 27 21:01:48.054288 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writer thread running {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 27 21:01:48.175001 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560357ce-fabe-46d8-bca5-10ed8b2353ee None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:01:48.179873 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0b23f6-2cf4-4c8a-9b6f-6c68fc82903d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:01:48.180565 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b23f6-2cf4-4c8a-9b6f-6c68fc82903d admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:48.180839 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b23f6-2cf4-4c8a-9b6f-6c68fc82903d admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:48.181145 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0b23f6-2cf4-4c8a-9b6f-6c68fc82903d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:48.191048 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0b23f6-2cf4-4c8a-9b6f-6c68fc82903d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:01:48.243416 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: WARNING keystone.common.rbac_enforcer.enforcer [None req-df0b23f6-2cf4-4c8a-9b6f-6c68fc82903d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 27 21:01:48.243651 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0b23f6-2cf4-4c8a-9b6f-6c68fc82903d admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:01:48.254571 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 2/4] 10.222.0.13 () {58 vars in 1324 bytes} [Mon Jun 27 21:01:47 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 21:01:48.261205 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717eac59-6132-4898-a5d5-758774d5b465 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:01:48.276266 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717eac59-6132-4898-a5d5-758774d5b465 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:01:48.281297 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f318b812-0f5e-4264-ab28-55a96fb9381d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:01:48.282469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318b812-0f5e-4264-ab28-55a96fb9381d admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:48.282752 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318b812-0f5e-4264-ab28-55a96fb9381d admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:48.283031 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318b812-0f5e-4264-ab28-55a96fb9381d admin admin] PATH_INFO: `/v3/projects/b6e5cf8b46f34e30a9b900f3cd1507c0` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:48.287265 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f318b812-0f5e-4264-ab28-55a96fb9381d admin admin] RBAC: Authorizing `identity:get_project(project_id=b6e5cf8b46f34e30a9b900f3cd1507c0)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:01:48.338664 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f318b812-0f5e-4264-ab28-55a96fb9381d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 27 21:01:48.339344 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f318b812-0f5e-4264-ab28-55a96fb9381d admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:01:48.343764 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 3/5] 10.222.0.13 () {56 vars in 1207 bytes} [Mon Jun 27 21:01:48 2022] GET /identity/v3/projects/b6e5cf8b46f34e30a9b900f3cd1507c0 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:01:50.361352 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad49cf4-8aa4-4b17-9c72-25cd53284b1b None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:50.362118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad49cf4-8aa4-4b17-9c72-25cd53284b1b None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:50.362118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad49cf4-8aa4-4b17-9c72-25cd53284b1b None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:50.369043 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 3/6] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:01:50 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:01:50.375340 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47be17-b91b-404b-b5af-816bfcf40f15 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:50.375340 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47be17-b91b-404b-b5af-816bfcf40f15 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:50.376358 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47be17-b91b-404b-b5af-816bfcf40f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:50.394788 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-8e47be17-b91b-404b-b5af-816bfcf40f15 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:01:50.396329 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-8e47be17-b91b-404b-b5af-816bfcf40f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:01:50.404998 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 4/7] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:01:50 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 27 21:01:50.427823 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad49cf4-8aa4-4b17-9c72-25cd53284b1b None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:01:50.446393 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad49cf4-8aa4-4b17-9c72-25cd53284b1b None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:01:50.453024 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5192d81a-5c88-46c0-babc-2bfe0764bc5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:01:50.453682 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5192d81a-5c88-46c0-babc-2bfe0764bc5f admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:50.453959 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5192d81a-5c88-46c0-babc-2bfe0764bc5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:50.454229 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5192d81a-5c88-46c0-babc-2bfe0764bc5f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:50.463891 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5192d81a-5c88-46c0-babc-2bfe0764bc5f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:01:50.465730 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5192d81a-5c88-46c0-babc-2bfe0764bc5f admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:01:50.475439 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 4/8] 10.222.0.13 () {58 vars in 1324 bytes} [Mon Jun 27 21:01:50 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 21:01:50.482712 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e47be17-b91b-404b-b5af-816bfcf40f15 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:01:50.492681 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e47be17-b91b-404b-b5af-816bfcf40f15 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:01:50.498793 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab9cac-c2a3-4cce-9c07-ca3d6dd5dd75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:01:50.499687 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab9cac-c2a3-4cce-9c07-ca3d6dd5dd75 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:50.499970 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab9cac-c2a3-4cce-9c07-ca3d6dd5dd75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:50.500246 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab9cac-c2a3-4cce-9c07-ca3d6dd5dd75 admin admin] PATH_INFO: `/v3/users/a9e7736617b74291b419604a9ff9d7e1` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:50.501856 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab9cac-c2a3-4cce-9c07-ca3d6dd5dd75 admin admin] RBAC: Authorizing `identity:get_user(user_id=a9e7736617b74291b419604a9ff9d7e1)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:01:50.503746 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab9cac-c2a3-4cce-9c07-ca3d6dd5dd75 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:01:50.505495 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 5/9] 10.222.0.13 () {56 vars in 1198 bytes} [Mon Jun 27 21:01:50 2022] GET /identity/v3/users/a9e7736617b74291b419604a9ff9d7e1 => generated 247 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:01:52.613945 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abbaf2b-404e-4458-b009-bd43752df22d None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:52.614493 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abbaf2b-404e-4458-b009-bd43752df22d None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:52.614493 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abbaf2b-404e-4458-b009-bd43752df22d None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:52.615860 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 5/10] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:01:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:01:52.621126 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011e98c-2089-4975-89b8-ad30eb024808 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:52.621635 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad791f1-81b1-4470-9591-07873af87c07 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:52.621719 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011e98c-2089-4975-89b8-ad30eb024808 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:52.621792 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad791f1-81b1-4470-9591-07873af87c07 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:52.621864 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011e98c-2089-4975-89b8-ad30eb024808 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:52.621948 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad791f1-81b1-4470-9591-07873af87c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:52.623095 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 6/11] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:01:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:01:52.628620 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd59edb-dec6-4314-971a-badea676b602 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:52.628909 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd59edb-dec6-4314-971a-badea676b602 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:52.629224 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd59edb-dec6-4314-971a-badea676b602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:52.646483 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-1bd59edb-dec6-4314-971a-badea676b602 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:01:52.647719 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-1bd59edb-dec6-4314-971a-badea676b602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:01:52.662416 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG passlib.handlers.bcrypt [None req-bad791f1-81b1-4470-9591-07873af87c07 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80389) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 27 21:01:52.662844 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG passlib.handlers.bcrypt [None req-bad791f1-81b1-4470-9591-07873af87c07 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80389) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 27 21:01:52.707857 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-bad791f1-81b1-4470-9591-07873af87c07 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:01:52.709525 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-bad791f1-81b1-4470-9591-07873af87c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:01:52.734918 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 7/12] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:01:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:01:52.757672 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd59edb-dec6-4314-971a-badea676b602 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:01:52.782695 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:01:52.815484 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 6/13] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:01:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:01:52.839164 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad791f1-81b1-4470-9591-07873af87c07 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:01:52.866424 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd59edb-dec6-4314-971a-badea676b602 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:01:52.901864 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279cf995-790e-4c1b-8223-8dc311c1fa3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:01:52.902724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279cf995-790e-4c1b-8223-8dc311c1fa3e None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:52.903145 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279cf995-790e-4c1b-8223-8dc311c1fa3e None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:52.903433 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279cf995-790e-4c1b-8223-8dc311c1fa3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:52.946589 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad791f1-81b1-4470-9591-07873af87c07 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:01:52.949692 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:01:52.985557 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279cf995-790e-4c1b-8223-8dc311c1fa3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:01:52.985557 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279cf995-790e-4c1b-8223-8dc311c1fa3e None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:01:52.986078 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d862050-f0c2-4af5-a184-74f53def9381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:01:52.987169 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d862050-f0c2-4af5-a184-74f53def9381 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:52.987458 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d862050-f0c2-4af5-a184-74f53def9381 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:52.987739 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d862050-f0c2-4af5-a184-74f53def9381 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:53.000702 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d862050-f0c2-4af5-a184-74f53def9381 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:01:53.005066 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:01:53.012302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-3d862050-f0c2-4af5-a184-74f53def9381 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:01:53.012302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:01:53.012302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:01:53.012302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:01:53.012302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:01:53.012302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:01:53.012302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:01:53.012302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:01:53.012302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:01:53.012302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:01:53.012302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:01:53.012302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:01:53.012302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:01:53.012302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:01:53.012302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:01:53.012302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:01:53.013276 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:01:53.013276 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:01:53.013276 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:01:53.013276 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:01:53.013276 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:01:53.013276 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:01:53.013276 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:01:53.013276 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:01:53.013276 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:01:53.013276 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:01:53.013276 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:01:53.013276 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:01:53.013276 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:01:53.013276 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:01:53.013276 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:01:53.013276 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:01:53.014063 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:01:53.014063 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:01:53.014063 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:01:53.014063 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:01:53.014063 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:01:53.014063 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:01:53.014063 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:01:53.014063 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:01:53.014063 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:01:53.014063 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:01:53.014063 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:01:53.014063 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:01:53.014063 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 7/14] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 27 21:01:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:01:53.019703 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d862050-f0c2-4af5-a184-74f53def9381 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:01:53.092506 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 8/15] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:01:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 21:01:53.098535 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279cf995-790e-4c1b-8223-8dc311c1fa3e None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:01:53.124093 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d862050-f0c2-4af5-a184-74f53def9381 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:01:53.159926 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63912a9f-2789-4547-8503-4118d1143d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:01:53.160610 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63912a9f-2789-4547-8503-4118d1143d0e None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:53.160896 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63912a9f-2789-4547-8503-4118d1143d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:53.161209 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63912a9f-2789-4547-8503-4118d1143d0e None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:53.161977 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63912a9f-2789-4547-8503-4118d1143d0e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:01:53.162286 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63912a9f-2789-4547-8503-4118d1143d0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:01:53.163753 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63912a9f-2789-4547-8503-4118d1143d0e None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:01:53.176360 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 8/16] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:01:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:01:53.181916 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63912a9f-2789-4547-8503-4118d1143d0e None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:01:53.209266 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:01:53.214029 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279cf995-790e-4c1b-8223-8dc311c1fa3e None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:01:53.238161 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:01:53.251838 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb17ab27-4404-4f8e-a26c-3175e12ca8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:01:53.252772 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17ab27-4404-4f8e-a26c-3175e12ca8ee None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:53.253147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17ab27-4404-4f8e-a26c-3175e12ca8ee None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:53.253496 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17ab27-4404-4f8e-a26c-3175e12ca8ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:53.265171 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb17ab27-4404-4f8e-a26c-3175e12ca8ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:01:53.266689 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb17ab27-4404-4f8e-a26c-3175e12ca8ee None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:01:53.286670 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63912a9f-2789-4547-8503-4118d1143d0e None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:01:53.288760 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-cb17ab27-4404-4f8e-a26c-3175e12ca8ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:01:53.288760 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:01:53.288760 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:01:53.288760 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:01:53.288760 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:01:53.288760 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:01:53.288760 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:01:53.288760 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:01:53.288760 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:01:53.288760 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:01:53.288760 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:01:53.288760 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:01:53.288760 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:01:53.288760 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:01:53.288760 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:01:53.289767 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:01:53.289767 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:01:53.289767 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:01:53.289767 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:01:53.289767 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:01:53.289767 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:01:53.289767 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:01:53.289767 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:01:53.289767 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:01:53.289767 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:01:53.289767 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:01:53.289767 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:01:53.289767 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:01:53.289767 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:01:53.289767 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:01:53.289767 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:01:53.289767 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:01:53.290957 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:01:53.290957 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:01:53.290957 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:01:53.290957 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:01:53.290957 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 9/17] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:01:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:01:53.296595 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb17ab27-4404-4f8e-a26c-3175e12ca8ee None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:01:53.327460 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded45037-915d-4b4e-b63f-f5f05ab3bda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:01:53.328026 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded45037-915d-4b4e-b63f-f5f05ab3bda7 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:53.328310 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded45037-915d-4b4e-b63f-f5f05ab3bda7 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:53.328585 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded45037-915d-4b4e-b63f-f5f05ab3bda7 None admin] PATH_INFO: `/v3/users/a9e7736617b74291b419604a9ff9d7e1` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:53.330306 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded45037-915d-4b4e-b63f-f5f05ab3bda7 None admin] RBAC: Authorizing `identity:get_user(user_id=a9e7736617b74291b419604a9ff9d7e1)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:01:53.331840 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded45037-915d-4b4e-b63f-f5f05ab3bda7 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:01:53.333954 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 9/18] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:01:53 2022] GET /identity/v3/users/a9e7736617b74291b419604a9ff9d7e1 => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:01:53.340116 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded45037-915d-4b4e-b63f-f5f05ab3bda7 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:01:53.402942 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb17ab27-4404-4f8e-a26c-3175e12ca8ee None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:01:53.437505 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:01:53.438566 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d041f2-0272-4db7-a2e9-5d377e8e0950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:01:53.440982 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d041f2-0272-4db7-a2e9-5d377e8e0950 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:53.441276 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d041f2-0272-4db7-a2e9-5d377e8e0950 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:53.441553 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d041f2-0272-4db7-a2e9-5d377e8e0950 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:53.442306 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d041f2-0272-4db7-a2e9-5d377e8e0950 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:01:53.443172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d041f2-0272-4db7-a2e9-5d377e8e0950 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:01:53.444590 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d041f2-0272-4db7-a2e9-5d377e8e0950 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:01:53.444806 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded45037-915d-4b4e-b63f-f5f05ab3bda7 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:01:53.467487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 10/19] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:01:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:01:53.482470 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc5dfec-673d-494f-94f2-9753a3cd4eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:01:53.483308 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5dfec-673d-494f-94f2-9753a3cd4eee None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:53.483591 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5dfec-673d-494f-94f2-9753a3cd4eee None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:53.483869 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc5dfec-673d-494f-94f2-9753a3cd4eee None admin] PATH_INFO: `/v3/domains/default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:53.485231 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc5dfec-673d-494f-94f2-9753a3cd4eee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:01:53.486751 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc5dfec-673d-494f-94f2-9753a3cd4eee None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:01:53.488481 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 10/20] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:01:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:01:53.494984 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d041f2-0272-4db7-a2e9-5d377e8e0950 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:01:53.531945 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:01:53.605451 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d041f2-0272-4db7-a2e9-5d377e8e0950 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:01:53.646185 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cab330-0b5f-4ac0-b34a-1a0afafe8883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:01:53.646864 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cab330-0b5f-4ac0-b34a-1a0afafe8883 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:53.647110 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cab330-0b5f-4ac0-b34a-1a0afafe8883 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:53.647374 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cab330-0b5f-4ac0-b34a-1a0afafe8883 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:53.648210 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cab330-0b5f-4ac0-b34a-1a0afafe8883 None admin] RBAC: Adding query filter params (role.id=d336dd7b746c4488b33289c00302e4d5, scope.domain.id=default, user.id=a9e7736617b74291b419604a9ff9d7e1) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:01:53.648518 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cab330-0b5f-4ac0-b34a-1a0afafe8883 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:01:53.650086 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cab330-0b5f-4ac0-b34a-1a0afafe8883 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:01:53.657684 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 11/21] 10.222.0.13 () {56 vars in 1322 bytes} [Mon Jun 27 21:01:53 2022] GET /identity/v3/role_assignments?user.id=a9e7736617b74291b419604a9ff9d7e1&scope.domain.id=default&role.id=d336dd7b746c4488b33289c00302e4d5 => generated 233 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:01:55.878287 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46839c9-860e-4454-b42e-98a4b5c52893 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:55.879211 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46839c9-860e-4454-b42e-98a4b5c52893 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:55.879775 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46839c9-860e-4454-b42e-98a4b5c52893 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:55.881367 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 11/22] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:01:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:01:55.888497 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33da205-6426-4cfd-9e35-f4667d842dd3 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:55.889448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33da205-6426-4cfd-9e35-f4667d842dd3 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:55.889946 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33da205-6426-4cfd-9e35-f4667d842dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:55.914015 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-f33da205-6426-4cfd-9e35-f4667d842dd3 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:01:55.915652 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-f33da205-6426-4cfd-9e35-f4667d842dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:01:56.007876 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 12/23] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:01:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:01:56.052112 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46839c9-860e-4454-b42e-98a4b5c52893 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:01:56.072317 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc4597-781f-4d76-a237-9cf4644ec56f None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:56.072981 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc4597-781f-4d76-a237-9cf4644ec56f None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:56.073494 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc4597-781f-4d76-a237-9cf4644ec56f None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:56.075146 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 13/24] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:01:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:01:56.080973 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb102e-da83-4677-a6cd-5e41213e5854 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:56.081594 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb102e-da83-4677-a6cd-5e41213e5854 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:56.082101 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb102e-da83-4677-a6cd-5e41213e5854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:56.108956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-56eb102e-da83-4677-a6cd-5e41213e5854 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:01:56.111359 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-56eb102e-da83-4677-a6cd-5e41213e5854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:01:56.165302 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:01:56.258138 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46839c9-860e-4454-b42e-98a4b5c52893 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:01:56.261253 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 14/25] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:01:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:01:56.262747 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a51a5-9f86-4778-b608-ebd8e1194259 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:56.263038 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a51a5-9f86-4778-b608-ebd8e1194259 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:56.263316 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a51a5-9f86-4778-b608-ebd8e1194259 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:56.264670 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 15/26] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:01:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:01:56.272125 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051ba6b-9d23-4e3c-b321-8d674b8a8ac8 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:56.272408 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051ba6b-9d23-4e3c-b321-8d674b8a8ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:56.272689 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051ba6b-9d23-4e3c-b321-8d674b8a8ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:56.309552 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-f051ba6b-9d23-4e3c-b321-8d674b8a8ac8 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:01:56.310798 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-f051ba6b-9d23-4e3c-b321-8d674b8a8ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:01:56.332856 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f648dfd1-da35-4b6d-9277-ec76cde84f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:01:56.333723 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648dfd1-da35-4b6d-9277-ec76cde84f9c None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:56.333987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648dfd1-da35-4b6d-9277-ec76cde84f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:56.334262 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648dfd1-da35-4b6d-9277-ec76cde84f9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:56.370269 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f648dfd1-da35-4b6d-9277-ec76cde84f9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:01:56.371810 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f648dfd1-da35-4b6d-9277-ec76cde84f9c None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:01:56.414970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-f648dfd1-da35-4b6d-9277-ec76cde84f9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:01:56.414970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:01:56.414970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:01:56.414970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:01:56.414970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:01:56.414970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:01:56.414970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:01:56.414970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:01:56.414970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:01:56.414970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:01:56.414970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:01:56.414970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:01:56.414970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:01:56.414970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:01:56.414970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:01:56.416015 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:01:56.416015 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:01:56.416015 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:01:56.416015 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:01:56.416015 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:01:56.416015 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:01:56.416015 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:01:56.416015 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:01:56.416015 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:01:56.416015 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:01:56.416015 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:01:56.416015 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:01:56.416015 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:01:56.416015 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:01:56.416015 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:01:56.416015 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:01:56.416015 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:01:56.416836 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:01:56.416836 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:01:56.416836 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:01:56.416836 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:01:56.416836 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 12/27] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:01:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:01:56.418892 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f648dfd1-da35-4b6d-9277-ec76cde84f9c None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:01:56.427409 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 16/28] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:01:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:01:56.428273 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f051ba6b-9d23-4e3c-b321-8d674b8a8ac8 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:01:56.455823 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:01:56.579252 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:01:56.594539 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f648dfd1-da35-4b6d-9277-ec76cde84f9c None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:01:56.595049 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f051ba6b-9d23-4e3c-b321-8d674b8a8ac8 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:01:56.634602 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468744e-32f8-4ca1-b0ff-34cfb36dc006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:01:56.635386 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468744e-32f8-4ca1-b0ff-34cfb36dc006 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:56.635576 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d71e852-716d-43a6-8619-4067bc6c0736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:01:56.635714 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468744e-32f8-4ca1-b0ff-34cfb36dc006 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:56.635936 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468744e-32f8-4ca1-b0ff-34cfb36dc006 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:56.636432 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71e852-716d-43a6-8619-4067bc6c0736 None admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:56.636691 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d468744e-32f8-4ca1-b0ff-34cfb36dc006 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:01:56.636804 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71e852-716d-43a6-8619-4067bc6c0736 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:56.637033 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d468744e-32f8-4ca1-b0ff-34cfb36dc006 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:01:56.637222 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71e852-716d-43a6-8619-4067bc6c0736 None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:56.638057 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d71e852-716d-43a6-8619-4067bc6c0736 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:01:56.638467 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d468744e-32f8-4ca1-b0ff-34cfb36dc006 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:01:56.639583 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d71e852-716d-43a6-8619-4067bc6c0736 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:01:56.652620 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 17/29] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:01:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:01:56.653476 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468744e-32f8-4ca1-b0ff-34cfb36dc006 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:01:56.771260 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468744e-32f8-4ca1-b0ff-34cfb36dc006 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:01:56.792464 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 13/30] 10.222.0.13 () {60 vars in 1131 bytes} [Mon Jun 27 21:01:56 2022] POST /identity/v3/roles => generated 219 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:01:56.794052 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def9ce8-7572-4316-a2ec-c8146aced6c1 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:56.794340 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def9ce8-7572-4316-a2ec-c8146aced6c1 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:56.794624 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def9ce8-7572-4316-a2ec-c8146aced6c1 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:56.796471 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 14/31] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:01:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:01:56.797355 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def9ce8-7572-4316-a2ec-c8146aced6c1 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:01:56.843022 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef6f43-7842-4d05-bdec-32d6fdf08a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:01:56.843725 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef6f43-7842-4d05-bdec-32d6fdf08a1b None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:56.844007 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef6f43-7842-4d05-bdec-32d6fdf08a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:56.844280 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef6f43-7842-4d05-bdec-32d6fdf08a1b None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:56.846695 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ef6f43-7842-4d05-bdec-32d6fdf08a1b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:01:56.848249 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ef6f43-7842-4d05-bdec-32d6fdf08a1b None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:01:56.974896 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:01:57.010890 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def9ce8-7572-4316-a2ec-c8146aced6c1 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:01:57.050611 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 18/32] 10.222.0.13 () {60 vars in 1131 bytes} [Mon Jun 27 21:01:56 2022] POST /identity/v3/roles => generated 225 bytes in 398 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:01:57.057171 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48e207-cb22-40d1-bb8b-0c14f0ff8317 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:57.057602 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48e207-cb22-40d1-bb8b-0c14f0ff8317 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:57.057890 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48e207-cb22-40d1-bb8b-0c14f0ff8317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:57.062034 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580b1cd-d103-4e42-9060-72b8d86a7caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:01:57.062754 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580b1cd-d103-4e42-9060-72b8d86a7caf None admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:57.063037 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580b1cd-d103-4e42-9060-72b8d86a7caf None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:57.063312 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580b1cd-d103-4e42-9060-72b8d86a7caf None admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:57.064089 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d580b1cd-d103-4e42-9060-72b8d86a7caf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:01:57.065847 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d580b1cd-d103-4e42-9060-72b8d86a7caf None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:01:57.097283 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-3f48e207-cb22-40d1-bb8b-0c14f0ff8317 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:01:57.099144 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-3f48e207-cb22-40d1-bb8b-0c14f0ff8317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:01:57.103647 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:01:57.136872 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 15/33] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Jun 27 21:01:56 2022] POST /identity/v3/projects => generated 301 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:01:57.219392 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 19/34] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:01:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:01:57.246437 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580b1cd-d103-4e42-9060-72b8d86a7caf None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:01:57.391787 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580b1cd-d103-4e42-9060-72b8d86a7caf None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:01:57.421627 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:01:57.432257 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82282bc-c031-42c4-a38d-e54c09cce55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:01:57.433145 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82282bc-c031-42c4-a38d-e54c09cce55b None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:57.433646 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82282bc-c031-42c4-a38d-e54c09cce55b None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:57.434100 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82282bc-c031-42c4-a38d-e54c09cce55b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:57.445680 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82282bc-c031-42c4-a38d-e54c09cce55b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:01:57.453934 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-e82282bc-c031-42c4-a38d-e54c09cce55b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:01:57.453934 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:01:57.453934 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:01:57.453934 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:01:57.453934 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:01:57.453934 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:01:57.453934 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:01:57.453934 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:01:57.453934 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:01:57.453934 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:01:57.453934 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:01:57.453934 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:01:57.453934 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:01:57.453934 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:01:57.453934 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:01:57.453934 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:01:57.454809 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:01:57.454809 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:01:57.454809 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:01:57.454809 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:01:57.454809 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:01:57.454809 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:01:57.454809 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:01:57.454809 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:01:57.454809 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:01:57.454809 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:01:57.454809 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:01:57.454809 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:01:57.454809 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:01:57.454809 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:01:57.454809 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:01:57.454809 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:01:57.455582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:01:57.455582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:01:57.455582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:01:57.455582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:01:57.455582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:01:57.455582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:01:57.455582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:01:57.455582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:01:57.455582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:01:57.455582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:01:57.455582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:01:57.455582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:01:57.455582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 16/35] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 27 21:01:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:01:57.461369 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48e207-cb22-40d1-bb8b-0c14f0ff8317 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:01:57.506341 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca8ad4-68bf-46a0-8615-3f749956ead8 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:57.506708 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca8ad4-68bf-46a0-8615-3f749956ead8 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:57.506914 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca8ad4-68bf-46a0-8615-3f749956ead8 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:57.508437 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 17/36] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:01:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:01:57.513472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe696689-31c6-4e6d-92db-1491780fcb19 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:57.513763 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe696689-31c6-4e6d-92db-1491780fcb19 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:57.514050 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe696689-31c6-4e6d-92db-1491780fcb19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:57.531654 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-fe696689-31c6-4e6d-92db-1491780fcb19 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:01:57.533229 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-fe696689-31c6-4e6d-92db-1491780fcb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:01:57.570035 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48e207-cb22-40d1-bb8b-0c14f0ff8317 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:01:57.600434 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 18/37] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:01:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:01:57.601609 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a4783-964b-4c1e-8291-ea5ea8e661f3 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:57.601904 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a4783-964b-4c1e-8291-ea5ea8e661f3 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:57.602180 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a4783-964b-4c1e-8291-ea5ea8e661f3 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:57.604381 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 19/38] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:01:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:01:57.609054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd0c7bb-f962-4c56-838a-daa0c4c8e69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:01:57.609512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6692e54-ccb9-494d-ac76-a9b8e5838d85 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:57.609735 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0c7bb-f962-4c56-838a-daa0c4c8e69b None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:57.609828 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6692e54-ccb9-494d-ac76-a9b8e5838d85 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:57.610022 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0c7bb-f962-4c56-838a-daa0c4c8e69b None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:57.610116 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6692e54-ccb9-494d-ac76-a9b8e5838d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:57.610349 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0c7bb-f962-4c56-838a-daa0c4c8e69b None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:57.611062 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd0c7bb-f962-4c56-838a-daa0c4c8e69b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:01:57.611365 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd0c7bb-f962-4c56-838a-daa0c4c8e69b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:01:57.612905 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd0c7bb-f962-4c56-838a-daa0c4c8e69b None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:01:57.622111 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:01:57.626214 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 20/39] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:01:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:01:57.627011 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd0c7bb-f962-4c56-838a-daa0c4c8e69b None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:01:57.629156 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-e6692e54-ccb9-494d-ac76-a9b8e5838d85 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:01:57.631354 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-e6692e54-ccb9-494d-ac76-a9b8e5838d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:01:57.703207 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 20/40] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:01:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:01:57.704219 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6692e54-ccb9-494d-ac76-a9b8e5838d85 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:01:57.734909 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd0c7bb-f962-4c56-838a-daa0c4c8e69b None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:01:57.757939 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:01:57.828158 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6692e54-ccb9-494d-ac76-a9b8e5838d85 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:01:57.874133 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d00e85-8f84-4f97-ac83-b179db9395f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:01:57.875150 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d00e85-8f84-4f97-ac83-b179db9395f5 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:57.875437 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d00e85-8f84-4f97-ac83-b179db9395f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:57.875705 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d00e85-8f84-4f97-ac83-b179db9395f5 None admin] PATH_INFO: `/v3/users/a9e7736617b74291b419604a9ff9d7e1` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:57.877128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d00e85-8f84-4f97-ac83-b179db9395f5 None admin] RBAC: Authorizing `identity:get_user(user_id=a9e7736617b74291b419604a9ff9d7e1)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:01:57.878869 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d00e85-8f84-4f97-ac83-b179db9395f5 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:01:57.880845 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 21/41] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:01:57 2022] GET /identity/v3/users/a9e7736617b74291b419604a9ff9d7e1 => generated 247 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:01:57.881799 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d00e85-8f84-4f97-ac83-b179db9395f5 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:01:57.881906 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8f3b15-0aae-4c00-b213-f28aa2fd216f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:01:57.882482 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f3b15-0aae-4c00-b213-f28aa2fd216f None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:57.882765 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f3b15-0aae-4c00-b213-f28aa2fd216f None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:57.883039 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f3b15-0aae-4c00-b213-f28aa2fd216f None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:57.883793 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8f3b15-0aae-4c00-b213-f28aa2fd216f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:01:57.885511 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8f3b15-0aae-4c00-b213-f28aa2fd216f None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:01:57.901495 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 21/42] 10.222.0.13 () {60 vars in 1131 bytes} [Mon Jun 27 21:01:57 2022] POST /identity/v3/roles => generated 223 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:01:57.901495 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8f3b15-0aae-4c00-b213-f28aa2fd216f None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:01:57.978858 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:01:58.029428 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d00e85-8f84-4f97-ac83-b179db9395f5 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:01:58.030328 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8f3b15-0aae-4c00-b213-f28aa2fd216f None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:01:58.050806 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:01:58.073060 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f637db7-7a5f-43cc-aaa9-6850faad520e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:01:58.073936 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f637db7-7a5f-43cc-aaa9-6850faad520e None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:58.074397 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f637db7-7a5f-43cc-aaa9-6850faad520e None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:58.074685 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f637db7-7a5f-43cc-aaa9-6850faad520e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:58.075920 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f637db7-7a5f-43cc-aaa9-6850faad520e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:01:58.077430 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f637db7-7a5f-43cc-aaa9-6850faad520e None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:01:58.078735 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff8614-8da6-4a61-8211-0c3fa5db3ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:01:58.078735 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff8614-8da6-4a61-8211-0c3fa5db3ffb None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:58.078929 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff8614-8da6-4a61-8211-0c3fa5db3ffb None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:58.080390 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff8614-8da6-4a61-8211-0c3fa5db3ffb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:58.080502 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 22/43] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:01:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:01:58.080586 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ff8614-8da6-4a61-8211-0c3fa5db3ffb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:01:58.081738 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ff8614-8da6-4a61-8211-0c3fa5db3ffb None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:01:58.083405 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 22/44] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:01:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:01:58.085394 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f637db7-7a5f-43cc-aaa9-6850faad520e None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:01:58.089659 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff8614-8da6-4a61-8211-0c3fa5db3ffb None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:01:58.192846 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f637db7-7a5f-43cc-aaa9-6850faad520e None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:01:58.198648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff8614-8da6-4a61-8211-0c3fa5db3ffb None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:01:58.219122 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:01:58.227098 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f3a765-e409-49d0-9942-99ddef936d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:01:58.228443 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f3a765-e409-49d0-9942-99ddef936d08 None admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:58.228957 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f3a765-e409-49d0-9942-99ddef936d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:58.229330 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f3a765-e409-49d0-9942-99ddef936d08 None admin] PATH_INFO: `/v3/domains/default/users/a9e7736617b74291b419604a9ff9d7e1/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:58.231374 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f3a765-e409-49d0-9942-99ddef936d08 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=a9e7736617b74291b419604a9ff9d7e1, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:01:58.232838 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f3a765-e409-49d0-9942-99ddef936d08 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:01:58.234323 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dd9cbe-7fb1-48e3-9f42-e9cb9a82f633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:01:58.234969 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd9cbe-7fb1-48e3-9f42-e9cb9a82f633 None admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:01:58.235255 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd9cbe-7fb1-48e3-9f42-e9cb9a82f633 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:01:58.235535 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd9cbe-7fb1-48e3-9f42-e9cb9a82f633 None admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:01:58.236252 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dd9cbe-7fb1-48e3-9f42-e9cb9a82f633 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:01:58.237716 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dd9cbe-7fb1-48e3-9f42-e9cb9a82f633 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:01:58.246736 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 23/45] 10.222.0.13 () {58 vars in 1361 bytes} [Mon Jun 27 21:01:58 2022] PUT /identity/v3/domains/default/users/a9e7736617b74291b419604a9ff9d7e1/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:01:58.264498 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 23/46] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Jun 27 21:01:58 2022] POST /identity/v3/projects => generated 312 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:02:00.285942 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370b293-45e3-4f3b-96f4-9e87a1dc8447 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:00.286456 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370b293-45e3-4f3b-96f4-9e87a1dc8447 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:00.286456 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370b293-45e3-4f3b-96f4-9e87a1dc8447 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:00.287862 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 24/47] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:00.293690 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b8ebb-90a4-4294-8165-180fe3c7f5b6 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:00.294137 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b8ebb-90a4-4294-8165-180fe3c7f5b6 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:00.294222 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b8ebb-90a4-4294-8165-180fe3c7f5b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:00.302856 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08088805-9791-4190-a4b6-40162c8457a2 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:00.303149 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08088805-9791-4190-a4b6-40162c8457a2 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:00.303430 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08088805-9791-4190-a4b6-40162c8457a2 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:00.304984 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 25/48] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:00.310259 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a862ec6-ab17-4c3d-b34a-fe6471dedd1b None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:00.310552 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a862ec6-ab17-4c3d-b34a-fe6471dedd1b None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:00.310834 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a862ec6-ab17-4c3d-b34a-fe6471dedd1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:00.313109 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-dd0b8ebb-90a4-4294-8165-180fe3c7f5b6 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:00.314333 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-dd0b8ebb-90a4-4294-8165-180fe3c7f5b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:00.328865 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-5a862ec6-ab17-4c3d-b34a-fe6471dedd1b None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:00.330109 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-5a862ec6-ab17-4c3d-b34a-fe6471dedd1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:00.345225 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:00.390233 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 24/49] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:00.403395 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 26/50] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:00.413255 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b8ebb-90a4-4294-8165-180fe3c7f5b6 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:00.426758 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a862ec6-ab17-4c3d-b34a-fe6471dedd1b None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:00.542003 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b8ebb-90a4-4294-8165-180fe3c7f5b6 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:00.546744 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a862ec6-ab17-4c3d-b34a-fe6471dedd1b None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:00.566862 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:00.581578 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d169030a-cfb9-4aa1-94ea-b13c662d41b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:00.582472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169030a-cfb9-4aa1-94ea-b13c662d41b5 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:00.582781 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169030a-cfb9-4aa1-94ea-b13c662d41b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:00.583061 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169030a-cfb9-4aa1-94ea-b13c662d41b5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:00.586572 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399013e9-bea0-444d-9c50-32aeacacb836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:00.587490 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399013e9-bea0-444d-9c50-32aeacacb836 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:00.587849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399013e9-bea0-444d-9c50-32aeacacb836 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:00.588139 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399013e9-bea0-444d-9c50-32aeacacb836 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:00.589459 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399013e9-bea0-444d-9c50-32aeacacb836 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:00.591082 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399013e9-bea0-444d-9c50-32aeacacb836 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:00.592988 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 27/51] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:02:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:00.593771 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d169030a-cfb9-4aa1-94ea-b13c662d41b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:00.599121 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399013e9-bea0-444d-9c50-32aeacacb836 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:00.601950 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-d169030a-cfb9-4aa1-94ea-b13c662d41b5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:02:00.601950 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:00.601950 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:00.601950 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:00.601950 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:00.601950 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:00.601950 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:00.601950 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:00.601950 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:00.601950 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:00.601950 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:00.601950 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:00.601950 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:00.601950 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:00.601950 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:00.601950 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:00.604164 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:00.604164 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:00.604164 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:00.604164 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:00.604164 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:00.604164 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:00.604164 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:00.604164 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:00.604164 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:00.604164 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:00.604164 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:00.604164 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:00.604164 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:00.604164 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:00.604164 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:00.604164 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:00.605098 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:00.605098 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:00.605098 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:00.605098 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:00.605098 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:00.605098 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:00.605098 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:00.605098 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:00.605098 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:00.605098 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:00.605098 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:02:00.605098 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:02:00.605098 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 25/52] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 27 21:02:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:00.609254 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d169030a-cfb9-4aa1-94ea-b13c662d41b5 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:00.635418 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:00.711908 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399013e9-bea0-444d-9c50-32aeacacb836 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:00.713887 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d169030a-cfb9-4aa1-94ea-b13c662d41b5 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:00.748417 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984406e7-9b99-4279-977c-94c07a7eeb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:00.748780 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e97974-a584-4a48-91d7-68c7cd9cccdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:00.749127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984406e7-9b99-4279-977c-94c07a7eeb99 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:00.749412 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984406e7-9b99-4279-977c-94c07a7eeb99 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:00.749559 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e97974-a584-4a48-91d7-68c7cd9cccdb None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:00.749685 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984406e7-9b99-4279-977c-94c07a7eeb99 None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:00.749878 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e97974-a584-4a48-91d7-68c7cd9cccdb None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:00.750186 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e97974-a584-4a48-91d7-68c7cd9cccdb None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:00.750438 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984406e7-9b99-4279-977c-94c07a7eeb99 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:00.750743 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984406e7-9b99-4279-977c-94c07a7eeb99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:00.750983 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e97974-a584-4a48-91d7-68c7cd9cccdb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:00.752136 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984406e7-9b99-4279-977c-94c07a7eeb99 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:00.752590 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e97974-a584-4a48-91d7-68c7cd9cccdb None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:00.760543 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 26/53] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:00.766055 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984406e7-9b99-4279-977c-94c07a7eeb99 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:00.797054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 28/54] 10.222.0.13 () {60 vars in 1140 bytes} [Mon Jun 27 21:02:00 2022] POST /identity/v3/projects => generated 298 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:02:00.866440 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:00.885866 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984406e7-9b99-4279-977c-94c07a7eeb99 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:00.935483 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86881af-0979-40b3-b62c-60506eb00696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:00.936472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86881af-0979-40b3-b62c-60506eb00696 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:00.936806 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86881af-0979-40b3-b62c-60506eb00696 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:00.937132 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86881af-0979-40b3-b62c-60506eb00696 None admin] PATH_INFO: `/v3/users/a9e7736617b74291b419604a9ff9d7e1` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:00.938458 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86881af-0979-40b3-b62c-60506eb00696 None admin] RBAC: Authorizing `identity:get_user(user_id=a9e7736617b74291b419604a9ff9d7e1)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:00.940163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86881af-0979-40b3-b62c-60506eb00696 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:00.942310 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 27/55] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:00 2022] GET /identity/v3/users/a9e7736617b74291b419604a9ff9d7e1 => generated 247 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:00.948519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e97974-a584-4a48-91d7-68c7cd9cccdb None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:00.997436 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:01.067079 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e97974-a584-4a48-91d7-68c7cd9cccdb None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:01.105025 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad3fdbd-bbd5-4985-9038-f577a76054cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:01.105967 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3fdbd-bbd5-4985-9038-f577a76054cb None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:01.106278 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3fdbd-bbd5-4985-9038-f577a76054cb None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:01.106579 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3fdbd-bbd5-4985-9038-f577a76054cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:01.107863 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad3fdbd-bbd5-4985-9038-f577a76054cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:01.109552 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad3fdbd-bbd5-4985-9038-f577a76054cb None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:01.111334 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 29/56] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:02:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:01.117313 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86881af-0979-40b3-b62c-60506eb00696 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:01.342682 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86881af-0979-40b3-b62c-60506eb00696 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:01.385145 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c51fdde-777a-4924-b646-3b8678c9d2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:01.385860 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51fdde-777a-4924-b646-3b8678c9d2d3 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:01.386153 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51fdde-777a-4924-b646-3b8678c9d2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:01.386435 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51fdde-777a-4924-b646-3b8678c9d2d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:01.387338 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c51fdde-777a-4924-b646-3b8678c9d2d3 None admin] RBAC: Adding query filter params (role.id=d336dd7b746c4488b33289c00302e4d5, scope.domain.id=default, user.id=a9e7736617b74291b419604a9ff9d7e1) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:01.387668 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c51fdde-777a-4924-b646-3b8678c9d2d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:01.389188 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c51fdde-777a-4924-b646-3b8678c9d2d3 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:01.399230 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 28/57] 10.222.0.13 () {56 vars in 1322 bytes} [Mon Jun 27 21:02:01 2022] GET /identity/v3/role_assignments?user.id=a9e7736617b74291b419604a9ff9d7e1&scope.domain.id=default&role.id=d336dd7b746c4488b33289c00302e4d5 => generated 530 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:02.845892 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56552a47-22b4-4af6-8e6d-37cb186e958e None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:02.846809 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56552a47-22b4-4af6-8e6d-37cb186e958e None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:02.846809 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56552a47-22b4-4af6-8e6d-37cb186e958e None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:02.849037 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 30/58] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:02.856011 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b8877-bca0-4aac-843d-b676ccc15d89 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:02.856686 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b8877-bca0-4aac-843d-b676ccc15d89 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:02.857070 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b8877-bca0-4aac-843d-b676ccc15d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:02.882493 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-e48b8877-bca0-4aac-843d-b676ccc15d89 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:02.883686 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-e48b8877-bca0-4aac-843d-b676ccc15d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:02.898271 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:02.952999 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 29/59] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:02.980771 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56552a47-22b4-4af6-8e6d-37cb186e958e None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:03.097393 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56552a47-22b4-4af6-8e6d-37cb186e958e None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:03.104633 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:03.133852 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad13b22-7e75-4ed9-a441-1b5ae13041c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:03.134954 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad13b22-7e75-4ed9-a441-1b5ae13041c1 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:03.135245 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad13b22-7e75-4ed9-a441-1b5ae13041c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:03.135519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad13b22-7e75-4ed9-a441-1b5ae13041c1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:03.136712 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad13b22-7e75-4ed9-a441-1b5ae13041c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:03.138199 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad13b22-7e75-4ed9-a441-1b5ae13041c1 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:03.139833 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 31/60] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:02:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:03.145601 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b8877-bca0-4aac-843d-b676ccc15d89 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:03.246899 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b8877-bca0-4aac-843d-b676ccc15d89 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:03.294651 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc27bf3-3e33-4a01-9868-4b9a2b56a6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:03.295326 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc27bf3-3e33-4a01-9868-4b9a2b56a6f1 None admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:03.295611 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc27bf3-3e33-4a01-9868-4b9a2b56a6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:03.295923 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc27bf3-3e33-4a01-9868-4b9a2b56a6f1 None admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:03.296659 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc27bf3-3e33-4a01-9868-4b9a2b56a6f1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:03.298245 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc27bf3-3e33-4a01-9868-4b9a2b56a6f1 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:03.326270 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 30/61] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Jun 27 21:02:03 2022] POST /identity/v3/users => generated 275 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:02:06.496603 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c57e7-ae25-48ad-b847-c83594c0bab9 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:06.497726 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c57e7-ae25-48ad-b847-c83594c0bab9 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:06.498276 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c57e7-ae25-48ad-b847-c83594c0bab9 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:06.500173 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 32/62] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:06.510425 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e05ece-c6c4-40a2-971e-031ed233270c None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:06.511334 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e05ece-c6c4-40a2-971e-031ed233270c None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:06.511870 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e05ece-c6c4-40a2-971e-031ed233270c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:06.520813 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:06.535038 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-09e05ece-c6c4-40a2-971e-031ed233270c None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:06.536730 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-09e05ece-c6c4-40a2-971e-031ed233270c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:06.635854 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 31/63] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:06.689345 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5c57e7-ae25-48ad-b847-c83594c0bab9 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:06.790278 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bb7e8-710b-4799-9b20-5e1eec34e46a None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:06.790930 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bb7e8-710b-4799-9b20-5e1eec34e46a None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:06.791428 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bb7e8-710b-4799-9b20-5e1eec34e46a None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:06.793276 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 32/64] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:02:06.799354 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758e76b-c05a-4162-9301-cf2b9a65f792 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:06.799949 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758e76b-c05a-4162-9301-cf2b9a65f792 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:06.800425 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758e76b-c05a-4162-9301-cf2b9a65f792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:06.832885 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-2758e76b-c05a-4162-9301-cf2b9a65f792 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:06.836498 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-2758e76b-c05a-4162-9301-cf2b9a65f792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:06.867323 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5c57e7-ae25-48ad-b847-c83594c0bab9 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:06.917195 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74997925-068c-4d69-94e1-15e91c34f203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:06.918536 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74997925-068c-4d69-94e1-15e91c34f203 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:06.919095 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74997925-068c-4d69-94e1-15e91c34f203 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:06.919589 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74997925-068c-4d69-94e1-15e91c34f203 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:06.935684 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74997925-068c-4d69-94e1-15e91c34f203 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:06.952533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-74997925-068c-4d69-94e1-15e91c34f203 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:06.952533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:06.952533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:06.952533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:06.952533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:06.952533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:06.952533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:06.952533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:06.952533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:06.952533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:06.952533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:06.952533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:06.952533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:06.952533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:06.952533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:06.952533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:06.953693 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:06.953693 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:06.953693 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:06.953693 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:06.953693 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:06.953693 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:06.953693 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:06.953693 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:06.953693 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:06.953693 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:06.953693 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:06.953693 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:06.953693 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:06.953693 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:06.953693 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:06.953693 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:06.954460 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:06.954460 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:06.954460 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:06.954460 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:06.954460 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:06.954460 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:06.954460 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:06.954460 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:06.954460 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:06.954460 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:06.954460 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:06.954460 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:02:06.954460 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 33/65] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:06.961529 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc408d-71ea-4757-b0eb-53089e801966 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:06.961529 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc408d-71ea-4757-b0eb-53089e801966 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:06.961529 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc408d-71ea-4757-b0eb-53089e801966 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:06.961529 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 34/66] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:06.961529 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdc408d-71ea-4757-b0eb-53089e801966 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:06.966446 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 33/67] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:06.967786 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c5a7c-ef67-4ba2-8653-17e7bfb3b1dd None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:06.968369 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c5a7c-ef67-4ba2-8653-17e7bfb3b1dd None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:06.968817 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c5a7c-ef67-4ba2-8653-17e7bfb3b1dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:06.991744 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-ce1c5a7c-ef67-4ba2-8653-17e7bfb3b1dd None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:06.993178 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-ce1c5a7c-ef67-4ba2-8653-17e7bfb3b1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:07.023326 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:07.079537 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:07.093034 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 34/68] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:07.094169 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c5a7c-ef67-4ba2-8653-17e7bfb3b1dd None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:07.131266 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdc408d-71ea-4757-b0eb-53089e801966 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:07.169079 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2267e5ab-1de2-48de-b2fd-f4782f0f12ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:07.169765 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267e5ab-1de2-48de-b2fd-f4782f0f12ec None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:07.170061 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267e5ab-1de2-48de-b2fd-f4782f0f12ec None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:07.170336 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267e5ab-1de2-48de-b2fd-f4782f0f12ec None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:07.171114 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2267e5ab-1de2-48de-b2fd-f4782f0f12ec None admin] RBAC: Adding query filter params (name=member) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:07.171426 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2267e5ab-1de2-48de-b2fd-f4782f0f12ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:07.172836 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2267e5ab-1de2-48de-b2fd-f4782f0f12ec None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:07.186893 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 35/69] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 27 21:02:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:07.186893 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2267e5ab-1de2-48de-b2fd-f4782f0f12ec None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:07.240351 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c5a7c-ef67-4ba2-8653-17e7bfb3b1dd None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:07.285543 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8765e8ef-8856-4388-8f3e-e494b6d188db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:07.286347 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765e8ef-8856-4388-8f3e-e494b6d188db None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:07.286606 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765e8ef-8856-4388-8f3e-e494b6d188db None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:07.286955 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765e8ef-8856-4388-8f3e-e494b6d188db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:07.294769 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:07.302674 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8765e8ef-8856-4388-8f3e-e494b6d188db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:07.303078 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2267e5ab-1de2-48de-b2fd-f4782f0f12ec None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:07.311615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-8765e8ef-8856-4388-8f3e-e494b6d188db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:02:07.311615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:07.311615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:07.311615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:07.311615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:07.311615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:07.311615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:07.311615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:07.311615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:07.311615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:07.311615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:07.311615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:07.311615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:07.311615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:07.311615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:07.311615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:07.312571 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:07.312571 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:07.312571 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:07.312571 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:07.312571 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:07.312571 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:07.312571 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:07.312571 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:07.312571 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:07.312571 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:07.312571 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:07.312571 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:07.312571 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:07.312571 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:07.312571 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:07.312571 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:07.313379 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:07.313379 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:07.313379 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:07.313379 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:07.313379 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:07.313379 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:07.313379 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:07.313379 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:07.313379 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:07.313379 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:07.313379 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:02:07.313379 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:02:07.313379 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 35/70] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 27 21:02:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:07.314780 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff704c1-f967-4835-b6ef-1a77e9cb48a3 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:07.314780 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff704c1-f967-4835-b6ef-1a77e9cb48a3 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:07.314780 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff704c1-f967-4835-b6ef-1a77e9cb48a3 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:07.319800 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 36/71] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:07.319800 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff704c1-f967-4835-b6ef-1a77e9cb48a3 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:07.350767 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76e3e1-6f20-4b57-b6d5-52c1858f80d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:07.351575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76e3e1-6f20-4b57-b6d5-52c1858f80d3 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:07.351961 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76e3e1-6f20-4b57-b6d5-52c1858f80d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:07.352234 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76e3e1-6f20-4b57-b6d5-52c1858f80d3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:07.366534 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a76e3e1-6f20-4b57-b6d5-52c1858f80d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:07.375701 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-2a76e3e1-6f20-4b57-b6d5-52c1858f80d3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:02:07.375701 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:07.375701 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:07.375701 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:07.375701 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:07.375701 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:07.375701 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:07.375701 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:07.375701 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:07.375701 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:07.375701 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:07.375701 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:07.375701 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:07.375701 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:07.375701 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:07.375701 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:07.376636 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:07.376636 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:07.376636 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:07.376636 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:07.376636 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:07.376636 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:07.376636 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:07.376636 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:07.376636 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:07.376636 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:07.376636 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:07.376636 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:07.376636 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:07.376636 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:07.376636 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:07.376636 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:07.377461 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:07.377461 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:07.377461 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:07.377461 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:07.377461 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:07.377461 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:07.377461 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:07.377461 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:07.377461 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:07.377461 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:07.377461 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:02:07.377461 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:02:07.377461 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 36/72] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 27 21:02:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:07.378096 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76e3e1-6f20-4b57-b6d5-52c1858f80d3 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:07.392964 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:07.447094 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff704c1-f967-4835-b6ef-1a77e9cb48a3 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:07.484413 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e6208c-5030-4425-b1e2-74516a8df065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:07.487860 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6208c-5030-4425-b1e2-74516a8df065 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:07.487860 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6208c-5030-4425-b1e2-74516a8df065 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:07.487860 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6208c-5030-4425-b1e2-74516a8df065 None admin] PATH_INFO: `/v3/users/ed277620276442838d05c774bf48c920` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:07.490704 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76e3e1-6f20-4b57-b6d5-52c1858f80d3 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:07.502456 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e6208c-5030-4425-b1e2-74516a8df065 None admin] RBAC: Authorizing `identity:get_user(user_id=ed277620276442838d05c774bf48c920)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:07.503939 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e6208c-5030-4425-b1e2-74516a8df065 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:07.505739 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 37/73] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:07 2022] GET /identity/v3/users/ed277620276442838d05c774bf48c920 => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:07.506960 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c4862-7f68-43cf-aeea-1b0b8b28bf53 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:07.507247 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c4862-7f68-43cf-aeea-1b0b8b28bf53 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:07.507563 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c4862-7f68-43cf-aeea-1b0b8b28bf53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:07.529335 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-2f0c4862-7f68-43cf-aeea-1b0b8b28bf53 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:07.531488 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-2f0c4862-7f68-43cf-aeea-1b0b8b28bf53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:07.538978 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc620c-99d0-4ebd-ad78-25fd4f2dfdc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:07.539699 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc620c-99d0-4ebd-ad78-25fd4f2dfdc3 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:07.539982 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc620c-99d0-4ebd-ad78-25fd4f2dfdc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:07.540256 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc620c-99d0-4ebd-ad78-25fd4f2dfdc3 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:07.541099 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdc620c-99d0-4ebd-ad78-25fd4f2dfdc3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:07.541420 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdc620c-99d0-4ebd-ad78-25fd4f2dfdc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:07.542890 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdc620c-99d0-4ebd-ad78-25fd4f2dfdc3 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:07.552267 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 37/74] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:07.553172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc620c-99d0-4ebd-ad78-25fd4f2dfdc3 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:07.573571 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:07.611276 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 38/75] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:07.612521 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0bcc3-715c-46ef-9fc6-25d89e94f732 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:07.612812 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0bcc3-715c-46ef-9fc6-25d89e94f732 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:07.613145 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0bcc3-715c-46ef-9fc6-25d89e94f732 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:07.617277 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 39/76] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:07.617277 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0bcc3-715c-46ef-9fc6-25d89e94f732 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:07.663416 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc620c-99d0-4ebd-ad78-25fd4f2dfdc3 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:07.666995 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:07.701402 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d44013-50b2-414b-87e2-c610ab8df3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:07.702077 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d44013-50b2-414b-87e2-c610ab8df3fa None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:07.702357 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d44013-50b2-414b-87e2-c610ab8df3fa None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:07.702631 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d44013-50b2-414b-87e2-c610ab8df3fa None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:07.703398 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d44013-50b2-414b-87e2-c610ab8df3fa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:07.703689 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d44013-50b2-414b-87e2-c610ab8df3fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:07.705421 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d44013-50b2-414b-87e2-c610ab8df3fa None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:07.715224 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 38/77] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:02:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:07.716025 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d44013-50b2-414b-87e2-c610ab8df3fa None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:07.733572 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0bcc3-715c-46ef-9fc6-25d89e94f732 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:07.768900 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883235bc-69e6-4b31-9e09-52d2b8dd04b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:07.769589 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883235bc-69e6-4b31-9e09-52d2b8dd04b4 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:07.769875 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883235bc-69e6-4b31-9e09-52d2b8dd04b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:07.770152 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883235bc-69e6-4b31-9e09-52d2b8dd04b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:07.801663 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:07.826162 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d44013-50b2-414b-87e2-c610ab8df3fa None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:07.844730 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883235bc-69e6-4b31-9e09-52d2b8dd04b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:07.846332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883235bc-69e6-4b31-9e09-52d2b8dd04b4 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:07.873474 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d837e9b-fa0a-4557-858e-bd6a7471678f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:07.874276 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d837e9b-fa0a-4557-858e-bd6a7471678f None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:07.874568 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d837e9b-fa0a-4557-858e-bd6a7471678f None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:07.874850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d837e9b-fa0a-4557-858e-bd6a7471678f None admin] PATH_INFO: `/v3/users/a9e7736617b74291b419604a9ff9d7e1` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:07.876089 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d837e9b-fa0a-4557-858e-bd6a7471678f None admin] RBAC: Authorizing `identity:get_user(user_id=a9e7736617b74291b419604a9ff9d7e1)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:07.877594 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d837e9b-fa0a-4557-858e-bd6a7471678f None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:07.879252 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 39/78] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:07 2022] GET /identity/v3/users/a9e7736617b74291b419604a9ff9d7e1 => generated 247 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:07.880454 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df53eba-95d9-4eef-b2d5-69d2f4dc2cc4 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:07.880764 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df53eba-95d9-4eef-b2d5-69d2f4dc2cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:07.881073 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df53eba-95d9-4eef-b2d5-69d2f4dc2cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:07.897278 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:07.901795 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-7df53eba-95d9-4eef-b2d5-69d2f4dc2cc4 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:07.903052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-7df53eba-95d9-4eef-b2d5-69d2f4dc2cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:07.964384 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 40/79] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:02:07.965384 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883235bc-69e6-4b31-9e09-52d2b8dd04b4 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:07.975770 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 40/80] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:07.976637 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df53eba-95d9-4eef-b2d5-69d2f4dc2cc4 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:08.055376 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:08.077144 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df53eba-95d9-4eef-b2d5-69d2f4dc2cc4 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:08.077456 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883235bc-69e6-4b31-9e09-52d2b8dd04b4 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:08.113549 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c791f91a-ec8c-4e66-a00e-631895ba85d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:08.113760 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca640f-3fa8-4a15-b926-3a2c4e367835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:08.114445 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca640f-3fa8-4a15-b926-3a2c4e367835 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:08.114574 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c791f91a-ec8c-4e66-a00e-631895ba85d3 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:08.114730 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca640f-3fa8-4a15-b926-3a2c4e367835 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:08.114820 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c791f91a-ec8c-4e66-a00e-631895ba85d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:08.115007 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca640f-3fa8-4a15-b926-3a2c4e367835 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:08.115095 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c791f91a-ec8c-4e66-a00e-631895ba85d3 None admin] PATH_INFO: `/v3/users/ed277620276442838d05c774bf48c920` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:08.116314 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c791f91a-ec8c-4e66-a00e-631895ba85d3 None admin] RBAC: Authorizing `identity:get_user(user_id=ed277620276442838d05c774bf48c920)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:08.117824 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c791f91a-ec8c-4e66-a00e-631895ba85d3 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:08.119498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 41/81] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:07 2022] GET /identity/v3/users/ed277620276442838d05c774bf48c920 => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:08.120645 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c791f91a-ec8c-4e66-a00e-631895ba85d3 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:08.128529 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ca640f-3fa8-4a15-b926-3a2c4e367835 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:08.137230 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-89ca640f-3fa8-4a15-b926-3a2c4e367835 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:08.137230 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:08.137230 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:08.137230 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:08.137230 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:08.137230 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:08.137230 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:08.137230 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:08.137230 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:08.137230 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:08.137230 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:08.137230 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:08.137230 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:08.137230 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:08.137230 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:08.137230 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:08.138104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:08.138104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:08.138104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:08.138104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:08.138104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:08.138104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:08.138104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:08.138104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:08.138104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:08.138104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:08.138104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:08.138104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:08.138104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:08.138104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:08.138104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:08.138104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:08.138877 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:08.138877 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:08.138877 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:08.138877 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:08.138877 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:08.138877 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:08.138877 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:08.138877 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:08.138877 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:08.138877 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:08.138877 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:08.138877 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:02:08.138877 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 41/82] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:08.139528 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca640f-3fa8-4a15-b926-3a2c4e367835 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:08.142370 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:08.222299 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c791f91a-ec8c-4e66-a00e-631895ba85d3 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:08.242123 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca640f-3fa8-4a15-b926-3a2c4e367835 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:08.256716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ce708-b404-4a00-9a3a-066f1b94aa1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:08.257409 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ce708-b404-4a00-9a3a-066f1b94aa1c None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:08.257693 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ce708-b404-4a00-9a3a-066f1b94aa1c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:08.257971 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ce708-b404-4a00-9a3a-066f1b94aa1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:08.274027 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:08.275931 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53e005-24cd-4143-890d-4a8bc6985a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:08.276862 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53e005-24cd-4143-890d-4a8bc6985a0a None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:08.277184 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53e005-24cd-4143-890d-4a8bc6985a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:08.277511 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53e005-24cd-4143-890d-4a8bc6985a0a None admin] PATH_INFO: `/v3/projects/04d09da302f44fefbe5b4502010ec050` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:08.278728 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a53e005-24cd-4143-890d-4a8bc6985a0a None admin] RBAC: Authorizing `identity:get_project(project_id=04d09da302f44fefbe5b4502010ec050)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:08.280238 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a53e005-24cd-4143-890d-4a8bc6985a0a None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:08.282001 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 42/83] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:08 2022] GET /identity/v3/projects/04d09da302f44fefbe5b4502010ec050 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:08.286596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53e005-24cd-4143-890d-4a8bc6985a0a None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:08.329094 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4ce708-b404-4a00-9a3a-066f1b94aa1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:08.330606 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4ce708-b404-4a00-9a3a-066f1b94aa1c None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:08.359659 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:08.390410 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a53e005-24cd-4143-890d-4a8bc6985a0a None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:08.424614 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06eeeff-d003-492f-a729-290d9645c04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:08.425495 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06eeeff-d003-492f-a729-290d9645c04e None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:08.425845 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06eeeff-d003-492f-a729-290d9645c04e None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:08.426148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06eeeff-d003-492f-a729-290d9645c04e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:08.427420 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06eeeff-d003-492f-a729-290d9645c04e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:08.429049 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06eeeff-d003-492f-a729-290d9645c04e None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:08.430792 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 43/84] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:02:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:08.435760 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 42/85] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:02:08.435903 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06eeeff-d003-492f-a729-290d9645c04e None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:08.435989 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ce708-b404-4a00-9a3a-066f1b94aa1c None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:08.484031 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:08.535274 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06eeeff-d003-492f-a729-290d9645c04e None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:08.539809 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ce708-b404-4a00-9a3a-066f1b94aa1c None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:08.569367 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48790a2-4ae8-4e70-8c8e-648db2c4e8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:08.570029 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48790a2-4ae8-4e70-8c8e-648db2c4e8ab None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:08.570317 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48790a2-4ae8-4e70-8c8e-648db2c4e8ab None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:08.570596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48790a2-4ae8-4e70-8c8e-648db2c4e8ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:08.574795 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c749b3d-6cc9-4cab-a77d-e4753e8fff8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:08.575432 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c749b3d-6cc9-4cab-a77d-e4753e8fff8b None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:08.575731 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c749b3d-6cc9-4cab-a77d-e4753e8fff8b None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:08.576026 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c749b3d-6cc9-4cab-a77d-e4753e8fff8b None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:08.576786 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c749b3d-6cc9-4cab-a77d-e4753e8fff8b None admin] RBAC: Adding query filter params (name=member) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:08.577128 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c749b3d-6cc9-4cab-a77d-e4753e8fff8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:08.578545 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c749b3d-6cc9-4cab-a77d-e4753e8fff8b None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:08.579446 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:08.586768 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 43/86] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 27 21:02:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:08.587590 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c749b3d-6cc9-4cab-a77d-e4753e8fff8b None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:08.642814 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48790a2-4ae8-4e70-8c8e-648db2c4e8ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:08.644223 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48790a2-4ae8-4e70-8c8e-648db2c4e8ab None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:08.693646 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c749b3d-6cc9-4cab-a77d-e4753e8fff8b None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:08.703215 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:08.730087 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdcc3f4-ce1d-4887-977e-9db789cff90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:08.730808 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdcc3f4-ce1d-4887-977e-9db789cff90b None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:08.731093 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdcc3f4-ce1d-4887-977e-9db789cff90b None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:08.731370 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdcc3f4-ce1d-4887-977e-9db789cff90b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:08.732211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdcc3f4-ce1d-4887-977e-9db789cff90b None admin] RBAC: Adding query filter params (role.id=e4414bde4df44de8aacb36c585a93e07, scope.project.id=04d09da302f44fefbe5b4502010ec050, user.id=ed277620276442838d05c774bf48c920) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:08.732526 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdcc3f4-ce1d-4887-977e-9db789cff90b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:08.734022 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdcc3f4-ce1d-4887-977e-9db789cff90b None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:08.739504 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 44/87] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:02:08 2022] GET /identity/v3/role_assignments?user.id=ed277620276442838d05c774bf48c920&scope.project.id=04d09da302f44fefbe5b4502010ec050&role.id=e4414bde4df44de8aacb36c585a93e07 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:08.740268 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdcc3f4-ce1d-4887-977e-9db789cff90b None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:08.754481 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 44/88] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:02:08.758496 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48790a2-4ae8-4e70-8c8e-648db2c4e8ab None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:08.805954 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:08.871690 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdcc3f4-ce1d-4887-977e-9db789cff90b None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:08.876720 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48790a2-4ae8-4e70-8c8e-648db2c4e8ab None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:08.922664 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c386b33-8243-4855-ac97-898f46981056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:08.922891 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a805822d-1215-428a-8fe1-042c49302f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:08.923523 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805822d-1215-428a-8fe1-042c49302f7f None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:08.923628 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c386b33-8243-4855-ac97-898f46981056 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:08.923803 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805822d-1215-428a-8fe1-042c49302f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:08.923917 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c386b33-8243-4855-ac97-898f46981056 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:08.924089 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805822d-1215-428a-8fe1-042c49302f7f None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:08.924176 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c386b33-8243-4855-ac97-898f46981056 None admin] PATH_INFO: `/v3/projects/04d09da302f44fefbe5b4502010ec050` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:08.924798 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a805822d-1215-428a-8fe1-042c49302f7f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:08.925325 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c386b33-8243-4855-ac97-898f46981056 None admin] RBAC: Authorizing `identity:get_project(project_id=04d09da302f44fefbe5b4502010ec050)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:08.926281 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a805822d-1215-428a-8fe1-042c49302f7f None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:08.926809 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c386b33-8243-4855-ac97-898f46981056 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:08.934221 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 45/89] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:08 2022] GET /identity/v3/projects/04d09da302f44fefbe5b4502010ec050 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:08.934221 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c386b33-8243-4855-ac97-898f46981056 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:08.954668 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 45/90] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Jun 27 21:02:08 2022] POST /identity/v3/users => generated 289 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:02:08.965368 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a805822d-1215-428a-8fe1-042c49302f7f None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:09.007715 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:09.074586 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c386b33-8243-4855-ac97-898f46981056 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:09.084443 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:09.117228 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a805822d-1215-428a-8fe1-042c49302f7f None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:09.117967 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e458184-14a0-4a79-90cc-5eb85b59c159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:09.119002 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e458184-14a0-4a79-90cc-5eb85b59c159 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:09.119292 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e458184-14a0-4a79-90cc-5eb85b59c159 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:09.119570 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e458184-14a0-4a79-90cc-5eb85b59c159 None admin] PATH_INFO: `/v3/users/ed277620276442838d05c774bf48c920` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:09.121063 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e458184-14a0-4a79-90cc-5eb85b59c159 None admin] RBAC: Authorizing `identity:get_user(user_id=ed277620276442838d05c774bf48c920)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:09.123021 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e458184-14a0-4a79-90cc-5eb85b59c159 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:09.125104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 46/91] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:08 2022] GET /identity/v3/users/ed277620276442838d05c774bf48c920 => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:09.125882 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e458184-14a0-4a79-90cc-5eb85b59c159 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:09.159596 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bb9178-7abd-45d6-8ea3-9172823e792c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:09.160475 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb9178-7abd-45d6-8ea3-9172823e792c None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:09.160756 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb9178-7abd-45d6-8ea3-9172823e792c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:09.161070 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb9178-7abd-45d6-8ea3-9172823e792c None admin] PATH_INFO: `/v3/projects/04d09da302f44fefbe5b4502010ec050` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:09.162272 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bb9178-7abd-45d6-8ea3-9172823e792c None admin] RBAC: Authorizing `identity:get_project(project_id=04d09da302f44fefbe5b4502010ec050)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:09.163755 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bb9178-7abd-45d6-8ea3-9172823e792c None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:09.165702 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 46/92] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:08 2022] GET /identity/v3/projects/04d09da302f44fefbe5b4502010ec050 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:09.166281 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bb9178-7abd-45d6-8ea3-9172823e792c None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:09.246596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e458184-14a0-4a79-90cc-5eb85b59c159 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:09.281751 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:09.287792 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456da4da-93dc-410d-8824-c219fded8be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:09.287965 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bb9178-7abd-45d6-8ea3-9172823e792c None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:09.288697 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456da4da-93dc-410d-8824-c219fded8be7 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:09.289509 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456da4da-93dc-410d-8824-c219fded8be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:09.289509 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456da4da-93dc-410d-8824-c219fded8be7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:09.290384 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456da4da-93dc-410d-8824-c219fded8be7 None admin] RBAC: Adding query filter params (role.id=d336dd7b746c4488b33289c00302e4d5, scope.project.id=04d09da302f44fefbe5b4502010ec050, user.id=a9e7736617b74291b419604a9ff9d7e1) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:09.290759 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456da4da-93dc-410d-8824-c219fded8be7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:09.293319 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456da4da-93dc-410d-8824-c219fded8be7 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:09.298743 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 47/93] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:02:09 2022] GET /identity/v3/role_assignments?user.id=a9e7736617b74291b419604a9ff9d7e1&scope.project.id=04d09da302f44fefbe5b4502010ec050&role.id=d336dd7b746c4488b33289c00302e4d5 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:09.299671 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456da4da-93dc-410d-8824-c219fded8be7 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:09.344333 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eee55e-f882-4cf4-b2c8-f19eafdd897a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:09.344668 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eee55e-f882-4cf4-b2c8-f19eafdd897a None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:09.345150 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eee55e-f882-4cf4-b2c8-f19eafdd897a None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:09.345483 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eee55e-f882-4cf4-b2c8-f19eafdd897a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:09.361700 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:09.445884 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eee55e-f882-4cf4-b2c8-f19eafdd897a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:09.448286 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eee55e-f882-4cf4-b2c8-f19eafdd897a None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:09.457871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456da4da-93dc-410d-8824-c219fded8be7 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:09.522838 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0f493d-f353-4b8d-8861-120862c32511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:09.523441 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f493d-f353-4b8d-8861-120862c32511 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:09.523727 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f493d-f353-4b8d-8861-120862c32511 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:09.524002 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f493d-f353-4b8d-8861-120862c32511 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:09.524865 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0f493d-f353-4b8d-8861-120862c32511 None admin] RBAC: Adding query filter params (role.id=5834784bf54d4806b88e32e0fb8e18fe, scope.project.id=04d09da302f44fefbe5b4502010ec050, user.id=ed277620276442838d05c774bf48c920) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:09.525965 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0f493d-f353-4b8d-8861-120862c32511 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:09.527522 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0f493d-f353-4b8d-8861-120862c32511 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:09.539691 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 48/94] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:02:09 2022] GET /identity/v3/role_assignments?user.id=ed277620276442838d05c774bf48c920&scope.project.id=04d09da302f44fefbe5b4502010ec050&role.id=5834784bf54d4806b88e32e0fb8e18fe => generated 259 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:09.593266 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:09.601737 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 47/95] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:02:09.608323 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0f493d-f353-4b8d-8861-120862c32511 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:09.762898 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0f493d-f353-4b8d-8861-120862c32511 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:09.764483 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:09.812091 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b322394d-d8b0-4afe-970d-8be2cadf1090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:09.812904 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322394d-d8b0-4afe-970d-8be2cadf1090 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:09.813216 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322394d-d8b0-4afe-970d-8be2cadf1090 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:09.813497 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322394d-d8b0-4afe-970d-8be2cadf1090 None admin] PATH_INFO: `/v3/projects/b2f7d83a41384ce5bfae26b46ae88e1f` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:09.814713 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b322394d-d8b0-4afe-970d-8be2cadf1090 None admin] RBAC: Authorizing `identity:get_project(project_id=b2f7d83a41384ce5bfae26b46ae88e1f)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:09.816193 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b322394d-d8b0-4afe-970d-8be2cadf1090 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:09.818136 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 49/96] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:09 2022] GET /identity/v3/projects/b2f7d83a41384ce5bfae26b46ae88e1f => generated 312 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:09.824222 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eee55e-f882-4cf4-b2c8-f19eafdd897a None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:09.962846 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eee55e-f882-4cf4-b2c8-f19eafdd897a None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:09.998925 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62211fc0-2529-4dbe-9c96-05874ab13b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:09.999625 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62211fc0-2529-4dbe-9c96-05874ab13b10 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:09.999933 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62211fc0-2529-4dbe-9c96-05874ab13b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:10.000235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62211fc0-2529-4dbe-9c96-05874ab13b10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:10.001160 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62211fc0-2529-4dbe-9c96-05874ab13b10 None admin] RBAC: Adding query filter params (role.id=e4414bde4df44de8aacb36c585a93e07, scope.project.id=b2f7d83a41384ce5bfae26b46ae88e1f, user.id=ed277620276442838d05c774bf48c920) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:10.001655 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62211fc0-2529-4dbe-9c96-05874ab13b10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:10.003209 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62211fc0-2529-4dbe-9c96-05874ab13b10 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:10.010674 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 48/97] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:02:09 2022] GET /identity/v3/role_assignments?user.id=ed277620276442838d05c774bf48c920&scope.project.id=b2f7d83a41384ce5bfae26b46ae88e1f&role.id=e4414bde4df44de8aacb36c585a93e07 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:11.427194 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b99ae-3b00-4b38-9b54-4e9e1d6d57f1 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:11.427771 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b99ae-3b00-4b38-9b54-4e9e1d6d57f1 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:11.427771 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b99ae-3b00-4b38-9b54-4e9e1d6d57f1 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:11.429186 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 50/98] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:11.434873 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50dfc0c-c655-44d1-b891-49caf59b6731 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:11.435373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50dfc0c-c655-44d1-b891-49caf59b6731 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:11.435453 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50dfc0c-c655-44d1-b891-49caf59b6731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:11.462057 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-d50dfc0c-c655-44d1-b891-49caf59b6731 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:11.463292 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d50dfc0c-c655-44d1-b891-49caf59b6731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:11.535292 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:11.553436 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 49/99] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:11.588720 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625b99ae-3b00-4b38-9b54-4e9e1d6d57f1 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:11.717806 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625b99ae-3b00-4b38-9b54-4e9e1d6d57f1 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:11.770197 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c3a095-9a62-4a1b-9487-92f95cd7b0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:11.771023 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3a095-9a62-4a1b-9487-92f95cd7b0a9 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:11.771313 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3a095-9a62-4a1b-9487-92f95cd7b0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:11.771597 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3a095-9a62-4a1b-9487-92f95cd7b0a9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:11.785779 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c3a095-9a62-4a1b-9487-92f95cd7b0a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:11.793842 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-f1c3a095-9a62-4a1b-9487-92f95cd7b0a9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:11.793842 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:11.793842 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:11.793842 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:11.793842 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:11.793842 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:11.793842 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:11.793842 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:11.793842 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:11.793842 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:11.793842 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:11.793842 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:11.793842 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:11.793842 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:11.793842 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:11.793842 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:11.795280 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:11.795280 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:11.795280 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:11.795280 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:11.795280 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:11.795280 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:11.795280 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:11.795280 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:11.795280 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:11.795280 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:11.795280 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:11.795280 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:11.795280 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:11.795280 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:11.795280 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:11.795280 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:11.796150 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:11.796150 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:11.796150 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:11.796150 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:11.796150 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:11.796150 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:11.796150 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:11.796150 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:11.796150 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:11.796150 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:11.796150 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:11.796150 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:02:11.796150 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 51/100] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:11.801651 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50dfc0c-c655-44d1-b891-49caf59b6731 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:11.939464 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50dfc0c-c655-44d1-b891-49caf59b6731 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:11.990342 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3893098f-dcf0-48ec-ada9-d264f7e62983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:11.991587 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893098f-dcf0-48ec-ada9-d264f7e62983 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:11.992407 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893098f-dcf0-48ec-ada9-d264f7e62983 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:11.992876 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893098f-dcf0-48ec-ada9-d264f7e62983 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:11.993880 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3893098f-dcf0-48ec-ada9-d264f7e62983 None admin] RBAC: Adding query filter params (name=member) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:11.994390 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3893098f-dcf0-48ec-ada9-d264f7e62983 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:11.996284 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3893098f-dcf0-48ec-ada9-d264f7e62983 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:11.999152 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8391e-de5b-4719-97d2-90742e6fa97a None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:11.999448 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8391e-de5b-4719-97d2-90742e6fa97a None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:12.000135 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8391e-de5b-4719-97d2-90742e6fa97a None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:12.001985 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 52/101] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:12.007921 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 50/102] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 27 21:02:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:12.013338 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea8391e-de5b-4719-97d2-90742e6fa97a None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:12.020916 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed18b7-689d-4099-9b12-724c20bbbe4f None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:12.021479 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed18b7-689d-4099-9b12-724c20bbbe4f None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:12.021961 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed18b7-689d-4099-9b12-724c20bbbe4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:12.040831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-7bed18b7-689d-4099-9b12-724c20bbbe4f None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:12.042599 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-7bed18b7-689d-4099-9b12-724c20bbbe4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:12.098419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:12.114431 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:12.131593 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 51/103] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:12.176475 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3218aee-1c0a-4aed-9360-93ac78c8c54e None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:12.177160 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3218aee-1c0a-4aed-9360-93ac78c8c54e None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:12.177656 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3218aee-1c0a-4aed-9360-93ac78c8c54e None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:12.179243 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 52/104] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:02:12.180140 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3218aee-1c0a-4aed-9360-93ac78c8c54e None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:12.274735 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea8391e-de5b-4719-97d2-90742e6fa97a None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:12.292181 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3218aee-1c0a-4aed-9360-93ac78c8c54e None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:12.318710 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af58d223-5629-406f-8bda-17f9c98896f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:12.319660 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58d223-5629-406f-8bda-17f9c98896f2 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:12.319963 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58d223-5629-406f-8bda-17f9c98896f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:12.320263 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58d223-5629-406f-8bda-17f9c98896f2 None admin] PATH_INFO: `/v3/users/ed277620276442838d05c774bf48c920` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:12.321594 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af58d223-5629-406f-8bda-17f9c98896f2 None admin] RBAC: Authorizing `identity:get_user(user_id=ed277620276442838d05c774bf48c920)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:12.323261 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af58d223-5629-406f-8bda-17f9c98896f2 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:12.324994 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 53/105] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:12 2022] GET /identity/v3/users/ed277620276442838d05c774bf48c920 => generated 275 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:12.326269 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f930b-e8f4-4f39-98c0-5fcbcd069f73 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:12.326578 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f930b-e8f4-4f39-98c0-5fcbcd069f73 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:12.326882 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f930b-e8f4-4f39-98c0-5fcbcd069f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:12.333695 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26e456-8b5d-4cc0-8604-2cc2ad72fdc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:12.334796 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26e456-8b5d-4cc0-8604-2cc2ad72fdc3 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:12.335284 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26e456-8b5d-4cc0-8604-2cc2ad72fdc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:12.335763 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26e456-8b5d-4cc0-8604-2cc2ad72fdc3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:12.344522 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-761f930b-e8f4-4f39-98c0-5fcbcd069f73 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:12.345862 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-761f930b-e8f4-4f39-98c0-5fcbcd069f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:12.351049 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a26e456-8b5d-4cc0-8604-2cc2ad72fdc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:12.368322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-0a26e456-8b5d-4cc0-8604-2cc2ad72fdc3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:02:12.368322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:12.368322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:12.368322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:12.368322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:12.368322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:12.368322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:12.368322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:12.368322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:12.368322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:12.368322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:12.368322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:12.368322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:12.368322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:12.368322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:12.368322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:12.369262 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:12.369262 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:12.369262 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:12.369262 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:12.369262 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:12.369262 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:12.369262 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:12.369262 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:12.369262 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:12.369262 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:12.369262 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:12.369262 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:12.369262 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:12.369262 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:12.369262 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:12.369262 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:12.370052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:12.370052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:12.370052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:12.370052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:12.370052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:12.370052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:12.370052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:12.370052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:12.370052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:12.370052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:12.370052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:02:12.370052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:02:12.370052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 53/106] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 27 21:02:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:12.371347 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26e456-8b5d-4cc0-8604-2cc2ad72fdc3 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:12.465333 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:12.469662 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 54/107] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:12.470588 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f930b-e8f4-4f39-98c0-5fcbcd069f73 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:12.527520 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:12.560434 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26e456-8b5d-4cc0-8604-2cc2ad72fdc3 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:12.607183 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa31aa11-0d3d-4091-b1ad-3fff602727b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:12.608488 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa31aa11-0d3d-4091-b1ad-3fff602727b2 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:12.608776 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa31aa11-0d3d-4091-b1ad-3fff602727b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:12.609345 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa31aa11-0d3d-4091-b1ad-3fff602727b2 None admin] PATH_INFO: `/v3/projects/04d09da302f44fefbe5b4502010ec050` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:12.610809 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa31aa11-0d3d-4091-b1ad-3fff602727b2 None admin] RBAC: Authorizing `identity:get_project(project_id=04d09da302f44fefbe5b4502010ec050)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:12.612964 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa31aa11-0d3d-4091-b1ad-3fff602727b2 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:12.614105 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f930b-e8f4-4f39-98c0-5fcbcd069f73 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:12.615046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 54/108] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:12 2022] GET /identity/v3/projects/04d09da302f44fefbe5b4502010ec050 => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:12.616105 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa31aa11-0d3d-4091-b1ad-3fff602727b2 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:12.656557 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e38c2-d0eb-45a7-86d8-2e9c6886c93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:12.657601 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e38c2-d0eb-45a7-86d8-2e9c6886c93d None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:12.657898 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e38c2-d0eb-45a7-86d8-2e9c6886c93d None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:12.658367 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e38c2-d0eb-45a7-86d8-2e9c6886c93d None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:12.659521 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7e38c2-d0eb-45a7-86d8-2e9c6886c93d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:12.659997 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7e38c2-d0eb-45a7-86d8-2e9c6886c93d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:12.661629 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7e38c2-d0eb-45a7-86d8-2e9c6886c93d None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:12.672999 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 55/109] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:12.674486 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b007bb75-cb16-4b25-be04-9df8cf8a3458 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:12.674707 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b007bb75-cb16-4b25-be04-9df8cf8a3458 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:12.675013 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b007bb75-cb16-4b25-be04-9df8cf8a3458 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:12.676332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 56/110] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:12.677116 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b007bb75-cb16-4b25-be04-9df8cf8a3458 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:12.753570 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa31aa11-0d3d-4091-b1ad-3fff602727b2 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:12.767118 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:12.801849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915ddf20-4357-42d2-a9a9-83014abb1445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:12.803083 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ddf20-4357-42d2-a9a9-83014abb1445 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:12.803427 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ddf20-4357-42d2-a9a9-83014abb1445 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:12.803725 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ddf20-4357-42d2-a9a9-83014abb1445 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:12.805735 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915ddf20-4357-42d2-a9a9-83014abb1445 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:12.807738 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915ddf20-4357-42d2-a9a9-83014abb1445 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:12.808531 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:12.813091 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 55/111] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:02:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:12.818315 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915ddf20-4357-42d2-a9a9-83014abb1445 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:12.827116 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b007bb75-cb16-4b25-be04-9df8cf8a3458 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:12.872289 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c495890a-cf0a-444d-8256-6e414689fb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:12.873382 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c495890a-cf0a-444d-8256-6e414689fb7d None admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:12.873665 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c495890a-cf0a-444d-8256-6e414689fb7d None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:12.873970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c495890a-cf0a-444d-8256-6e414689fb7d None admin] PATH_INFO: `/v3/projects/04d09da302f44fefbe5b4502010ec050/users/ed277620276442838d05c774bf48c920/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:12.876162 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c495890a-cf0a-444d-8256-6e414689fb7d None admin] RBAC: Authorizing `identity:create_grant(project_id=04d09da302f44fefbe5b4502010ec050, user_id=ed277620276442838d05c774bf48c920, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:12.877832 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c495890a-cf0a-444d-8256-6e414689fb7d None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:12.898159 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 57/112] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 27 21:02:12 2022] PUT /identity/v3/projects/04d09da302f44fefbe5b4502010ec050/users/ed277620276442838d05c774bf48c920/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:02:12.901349 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1978f80-05ed-462e-83e8-e31706cde48f None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:12.901851 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1978f80-05ed-462e-83e8-e31706cde48f None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:12.902241 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1978f80-05ed-462e-83e8-e31706cde48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:12.944127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-e1978f80-05ed-462e-83e8-e31706cde48f None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:12.947246 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-e1978f80-05ed-462e-83e8-e31706cde48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:13.024147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915ddf20-4357-42d2-a9a9-83014abb1445 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:13.080277 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7818a539-971f-47a5-89f6-c2e4c952589e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:13.081303 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7818a539-971f-47a5-89f6-c2e4c952589e None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:13.081659 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7818a539-971f-47a5-89f6-c2e4c952589e None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:13.082035 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7818a539-971f-47a5-89f6-c2e4c952589e None admin] PATH_INFO: `/v3/users/a9e7736617b74291b419604a9ff9d7e1` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:13.083418 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7818a539-971f-47a5-89f6-c2e4c952589e None admin] RBAC: Authorizing `identity:get_user(user_id=a9e7736617b74291b419604a9ff9d7e1)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:13.085102 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7818a539-971f-47a5-89f6-c2e4c952589e None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:13.087715 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 56/113] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:12 2022] GET /identity/v3/users/a9e7736617b74291b419604a9ff9d7e1 => generated 247 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:13.088430 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7818a539-971f-47a5-89f6-c2e4c952589e None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:13.107057 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 58/114] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:13.108027 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1978f80-05ed-462e-83e8-e31706cde48f None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:13.158963 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:13.213742 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:13.218830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7818a539-971f-47a5-89f6-c2e4c952589e None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:13.236412 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1978f80-05ed-462e-83e8-e31706cde48f None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:13.255348 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b1080e-3a8c-49f6-a1f1-35411b0a5078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:13.256260 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1080e-3a8c-49f6-a1f1-35411b0a5078 None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:13.256687 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1080e-3a8c-49f6-a1f1-35411b0a5078 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:13.257165 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b1080e-3a8c-49f6-a1f1-35411b0a5078 None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:13.257885 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b1080e-3a8c-49f6-a1f1-35411b0a5078 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:13.259349 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b1080e-3a8c-49f6-a1f1-35411b0a5078 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:13.272522 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f582589e-5550-4834-8543-ca195b6af28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:13.273436 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582589e-5550-4834-8543-ca195b6af28d None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:13.273727 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582589e-5550-4834-8543-ca195b6af28d None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:13.274005 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582589e-5550-4834-8543-ca195b6af28d None admin] PATH_INFO: `/v3/projects/04d09da302f44fefbe5b4502010ec050` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:13.275271 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f582589e-5550-4834-8543-ca195b6af28d None admin] RBAC: Authorizing `identity:get_project(project_id=04d09da302f44fefbe5b4502010ec050)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:13.276824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f582589e-5550-4834-8543-ca195b6af28d None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:13.279042 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 59/115] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:13 2022] GET /identity/v3/projects/04d09da302f44fefbe5b4502010ec050 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:13.280240 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b444f8-25eb-4e44-bcc0-94d8d100b7f5 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:13.280531 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b444f8-25eb-4e44-bcc0-94d8d100b7f5 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:13.280811 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b444f8-25eb-4e44-bcc0-94d8d100b7f5 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:13.284990 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 60/116] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:13.284990 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b444f8-25eb-4e44-bcc0-94d8d100b7f5 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:13.295309 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 57/117] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Jun 27 21:02:13 2022] POST /identity/v3/projects => generated 302 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:02:13.296645 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae23b2c-b291-4c7a-99df-10907e2ba751 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:13.296960 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae23b2c-b291-4c7a-99df-10907e2ba751 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:13.297288 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae23b2c-b291-4c7a-99df-10907e2ba751 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:13.298822 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 58/118] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:13.299625 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae23b2c-b291-4c7a-99df-10907e2ba751 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:13.421001 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b444f8-25eb-4e44-bcc0-94d8d100b7f5 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:13.452773 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:13.460911 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae23b2c-b291-4c7a-99df-10907e2ba751 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:13.467055 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a09b15-a228-4eb9-a3a4-0952b082fc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:13.472878 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a09b15-a228-4eb9-a3a4-0952b082fc9f None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:13.472878 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a09b15-a228-4eb9-a3a4-0952b082fc9f None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:13.472878 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a09b15-a228-4eb9-a3a4-0952b082fc9f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:13.487239 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a09b15-a228-4eb9-a3a4-0952b082fc9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:13.497262 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-b9a09b15-a228-4eb9-a3a4-0952b082fc9f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 21:02:13.497262 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:13.497262 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:13.497262 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:13.497262 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:13.497262 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:13.497262 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:13.497262 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:13.497262 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:13.497262 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:13.497262 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:13.497262 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:13.497262 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:13.497262 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:13.497262 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:13.497262 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:13.498205 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:13.498205 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:13.498205 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:13.498205 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:13.498205 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:13.498205 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:13.498205 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:13.498205 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:13.498205 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:13.498205 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:13.498205 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:13.498205 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:13.498205 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:13.498205 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:13.498205 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:13.498205 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:13.498974 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:13.498974 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:13.498974 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:13.498974 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:13.498974 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:13.498974 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:13.498974 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:13.498974 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:13.498974 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:13.498974 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:13.498974 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 21:02:13.498974 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:02:13.501508 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 61/119] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:13.503276 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc4ae5-4fa0-4996-97cc-f5123ef7d30c None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:13.503804 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc4ae5-4fa0-4996-97cc-f5123ef7d30c None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:13.504347 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc4ae5-4fa0-4996-97cc-f5123ef7d30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:13.508181 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf6a677-ae43-4a69-a130-d9dce03c37a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:13.511259 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6a677-ae43-4a69-a130-d9dce03c37a8 None admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:13.511546 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6a677-ae43-4a69-a130-d9dce03c37a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:13.511871 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6a677-ae43-4a69-a130-d9dce03c37a8 None admin] PATH_INFO: `/v3/projects/04d09da302f44fefbe5b4502010ec050/users/a9e7736617b74291b419604a9ff9d7e1/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:13.514968 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf6a677-ae43-4a69-a130-d9dce03c37a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=04d09da302f44fefbe5b4502010ec050, user_id=a9e7736617b74291b419604a9ff9d7e1, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:13.516511 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf6a677-ae43-4a69-a130-d9dce03c37a8 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:13.526041 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-4dcc4ae5-4fa0-4996-97cc-f5123ef7d30c None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:13.527804 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-4dcc4ae5-4fa0-4996-97cc-f5123ef7d30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:13.538197 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:13.541901 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 59/120] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 27 21:02:13 2022] PUT /identity/v3/projects/04d09da302f44fefbe5b4502010ec050/users/a9e7736617b74291b419604a9ff9d7e1/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:02:13.543700 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d38a8-01df-4340-a60f-60f6af76bbba None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:13.544447 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d38a8-01df-4340-a60f-60f6af76bbba None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:13.549077 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d38a8-01df-4340-a60f-60f6af76bbba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:13.594903 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-787d38a8-01df-4340-a60f-60f6af76bbba None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:13.599965 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-787d38a8-01df-4340-a60f-60f6af76bbba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:13.647383 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 62/121] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:13.648328 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcc4ae5-4fa0-4996-97cc-f5123ef7d30c None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:13.788872 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcc4ae5-4fa0-4996-97cc-f5123ef7d30c None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:13.795059 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 60/122] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:13.795957 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d38a8-01df-4340-a60f-60f6af76bbba None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:13.831644 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4499c2e1-35c7-4642-a4d5-148a8729eb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:13.832428 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499c2e1-35c7-4642-a4d5-148a8729eb7c None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:13.832720 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499c2e1-35c7-4642-a4d5-148a8729eb7c None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:13.833123 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499c2e1-35c7-4642-a4d5-148a8729eb7c None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:13.833902 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4499c2e1-35c7-4642-a4d5-148a8729eb7c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:13.834216 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4499c2e1-35c7-4642-a4d5-148a8729eb7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:13.835673 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4499c2e1-35c7-4642-a4d5-148a8729eb7c None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:13.844797 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 63/123] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 27 21:02:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:13.845511 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:13.850875 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4499c2e1-35c7-4642-a4d5-148a8729eb7c None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:13.959827 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4499c2e1-35c7-4642-a4d5-148a8729eb7c None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:13.980739 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:13.998301 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d38a8-01df-4340-a60f-60f6af76bbba None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:14.000418 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0064e36-a59d-403d-9a3b-522f41d02604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:14.001595 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0064e36-a59d-403d-9a3b-522f41d02604 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:14.002026 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0064e36-a59d-403d-9a3b-522f41d02604 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:14.002586 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0064e36-a59d-403d-9a3b-522f41d02604 None admin] PATH_INFO: `/v3/users/a40da6e662ee449cb5dd0f29b2025ab5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:14.021169 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0064e36-a59d-403d-9a3b-522f41d02604 None admin] RBAC: Authorizing `identity:get_user(user_id=a40da6e662ee449cb5dd0f29b2025ab5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:14.022875 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0064e36-a59d-403d-9a3b-522f41d02604 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:14.031566 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 64/124] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:13 2022] GET /identity/v3/users/a40da6e662ee449cb5dd0f29b2025ab5 => generated 289 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:14.032638 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0064e36-a59d-403d-9a3b-522f41d02604 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:14.082285 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b693d25-c29f-47a7-80bf-621b51f791dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:14.083716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b693d25-c29f-47a7-80bf-621b51f791dd None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:14.084333 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b693d25-c29f-47a7-80bf-621b51f791dd None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:14.084779 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b693d25-c29f-47a7-80bf-621b51f791dd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:14.099046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b693d25-c29f-47a7-80bf-621b51f791dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:14.109426 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-1b693d25-c29f-47a7-80bf-621b51f791dd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:02:14.109426 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:14.109426 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:14.109426 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:14.109426 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:14.109426 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:14.109426 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:14.109426 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:14.109426 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:14.109426 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:14.109426 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:14.109426 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:14.109426 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:14.109426 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:14.109426 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:14.109426 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:14.110346 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:14.110346 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:14.110346 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:14.110346 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:14.110346 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:14.110346 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:14.110346 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:14.110346 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:14.110346 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:14.110346 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:14.110346 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:14.110346 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:14.110346 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:14.110346 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:14.110346 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:14.110346 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:14.111139 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:14.111139 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:14.111139 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:14.111139 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:14.111139 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:14.111139 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:14.111139 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:14.111139 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:14.111139 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:14.111139 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:14.111139 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:02:14.111139 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:02:14.113974 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 61/125] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 27 21:02:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:14.115633 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b693d25-c29f-47a7-80bf-621b51f791dd None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:14.156689 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:14.249659 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0064e36-a59d-403d-9a3b-522f41d02604 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:14.277034 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b693d25-c29f-47a7-80bf-621b51f791dd None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:14.317888 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e562d874-5160-46d3-b647-6a948cecd486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:14.318916 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562d874-5160-46d3-b647-6a948cecd486 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:14.319419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562d874-5160-46d3-b647-6a948cecd486 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:14.319884 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562d874-5160-46d3-b647-6a948cecd486 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:14.323512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe29c4-a9db-4d54-9f52-97e9c7cdd4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:14.324444 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe29c4-a9db-4d54-9f52-97e9c7cdd4c8 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:14.324776 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe29c4-a9db-4d54-9f52-97e9c7cdd4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:14.325083 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe29c4-a9db-4d54-9f52-97e9c7cdd4c8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:14.343190 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbe29c4-a9db-4d54-9f52-97e9c7cdd4c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:14.353295 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:14.354142 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-ddbe29c4-a9db-4d54-9f52-97e9c7cdd4c8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:14.354142 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:14.354142 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:14.354142 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:14.354142 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:14.354142 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:14.354142 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:14.354142 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:14.354142 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:14.354142 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:14.354142 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:14.354142 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:14.354142 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:14.354142 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:14.354142 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:14.354142 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:14.355107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:14.355107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:14.355107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:14.355107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:14.355107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:14.355107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:14.355107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:14.355107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:14.355107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:14.355107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:14.355107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:14.355107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:14.355107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:14.355107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:14.355107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:14.355107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:14.356018 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:14.356018 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:14.356018 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:14.356018 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:14.356018 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:14.356018 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:14.356018 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:14.356018 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:14.356018 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:14.356018 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:14.356018 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:14.356018 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:02:14.360864 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 65/126] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:14.361748 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe29c4-a9db-4d54-9f52-97e9c7cdd4c8 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:14.430816 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e562d874-5160-46d3-b647-6a948cecd486 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:14.432424 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e562d874-5160-46d3-b647-6a948cecd486 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:14.526015 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe29c4-a9db-4d54-9f52-97e9c7cdd4c8 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:14.545951 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:14.549542 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 62/127] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:02:14.550471 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e562d874-5160-46d3-b647-6a948cecd486 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:14.581432 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c39795-1c33-42e6-84d8-e493daab0d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:14.582170 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c39795-1c33-42e6-84d8-e493daab0d11 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:14.582478 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c39795-1c33-42e6-84d8-e493daab0d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:14.582767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c39795-1c33-42e6-84d8-e493daab0d11 None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:14.583700 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c39795-1c33-42e6-84d8-e493daab0d11 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:14.584039 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c39795-1c33-42e6-84d8-e493daab0d11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:14.585699 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c39795-1c33-42e6-84d8-e493daab0d11 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:14.606626 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:14.611041 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 66/128] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:02:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:14.612063 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c39795-1c33-42e6-84d8-e493daab0d11 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:14.656972 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e562d874-5160-46d3-b647-6a948cecd486 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:14.699082 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f10829d-1961-4387-b7ee-1aec3ee4cdb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:14.700320 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10829d-1961-4387-b7ee-1aec3ee4cdb1 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:14.700749 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10829d-1961-4387-b7ee-1aec3ee4cdb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:14.701165 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10829d-1961-4387-b7ee-1aec3ee4cdb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:14.702124 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f10829d-1961-4387-b7ee-1aec3ee4cdb1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:14.702549 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f10829d-1961-4387-b7ee-1aec3ee4cdb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:14.704267 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f10829d-1961-4387-b7ee-1aec3ee4cdb1 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:14.714679 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 63/129] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 27 21:02:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:14.715763 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f10829d-1961-4387-b7ee-1aec3ee4cdb1 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:14.816211 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c39795-1c33-42e6-84d8-e493daab0d11 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:14.832988 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f10829d-1961-4387-b7ee-1aec3ee4cdb1 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:14.855297 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948d9b2-af4d-4e8d-957b-b80eb10bff33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:14.856270 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948d9b2-af4d-4e8d-957b-b80eb10bff33 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:14.856559 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948d9b2-af4d-4e8d-957b-b80eb10bff33 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:14.856895 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948d9b2-af4d-4e8d-957b-b80eb10bff33 None admin] PATH_INFO: `/v3/projects/04d09da302f44fefbe5b4502010ec050` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:14.858123 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4948d9b2-af4d-4e8d-957b-b80eb10bff33 None admin] RBAC: Authorizing `identity:get_project(project_id=04d09da302f44fefbe5b4502010ec050)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:14.859650 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4948d9b2-af4d-4e8d-957b-b80eb10bff33 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:14.861510 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 67/130] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:14 2022] GET /identity/v3/projects/04d09da302f44fefbe5b4502010ec050 => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:14.862418 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948d9b2-af4d-4e8d-957b-b80eb10bff33 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:14.873310 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:14.912567 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa854b-a1d4-4439-87cb-b22d491d356c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:14.913648 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa854b-a1d4-4439-87cb-b22d491d356c None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:14.913952 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa854b-a1d4-4439-87cb-b22d491d356c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:14.914251 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa854b-a1d4-4439-87cb-b22d491d356c None admin] PATH_INFO: `/v3/users/ed277620276442838d05c774bf48c920` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:14.915511 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:14.916348 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa854b-a1d4-4439-87cb-b22d491d356c None admin] RBAC: Authorizing `identity:get_user(user_id=ed277620276442838d05c774bf48c920)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:14.918221 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa854b-a1d4-4439-87cb-b22d491d356c None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:14.920091 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 64/131] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:14 2022] GET /identity/v3/users/ed277620276442838d05c774bf48c920 => generated 275 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:14.920958 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa854b-a1d4-4439-87cb-b22d491d356c None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:14.970547 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948d9b2-af4d-4e8d-957b-b80eb10bff33 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:15.005940 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c19009a-68a0-4326-a5d5-fa414ed1da69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:15.006835 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19009a-68a0-4326-a5d5-fa414ed1da69 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:15.007181 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19009a-68a0-4326-a5d5-fa414ed1da69 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:15.007463 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19009a-68a0-4326-a5d5-fa414ed1da69 None admin] PATH_INFO: `/v3/users/ed277620276442838d05c774bf48c920` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:15.008740 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c19009a-68a0-4326-a5d5-fa414ed1da69 None admin] RBAC: Authorizing `identity:get_user(user_id=ed277620276442838d05c774bf48c920)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:15.010287 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c19009a-68a0-4326-a5d5-fa414ed1da69 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:15.012229 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 68/132] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:14 2022] GET /identity/v3/users/ed277620276442838d05c774bf48c920 => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:15.015984 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c19009a-68a0-4326-a5d5-fa414ed1da69 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:15.129446 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa854b-a1d4-4439-87cb-b22d491d356c None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:15.149880 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c19009a-68a0-4326-a5d5-fa414ed1da69 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:15.169738 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:15.190035 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef193607-0c39-4b48-aae8-378544881761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:15.191258 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef193607-0c39-4b48-aae8-378544881761 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:15.191728 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef193607-0c39-4b48-aae8-378544881761 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:15.192132 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef193607-0c39-4b48-aae8-378544881761 None admin] PATH_INFO: `/v3/projects/04d09da302f44fefbe5b4502010ec050` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:15.193926 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef193607-0c39-4b48-aae8-378544881761 None admin] RBAC: Authorizing `identity:get_project(project_id=04d09da302f44fefbe5b4502010ec050)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:15.195675 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef193607-0c39-4b48-aae8-378544881761 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:15.197723 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 69/133] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:15 2022] GET /identity/v3/projects/04d09da302f44fefbe5b4502010ec050 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:15.198766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef193607-0c39-4b48-aae8-378544881761 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:15.199463 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99a91e0-3f0a-4d22-a3b1-d6b2ed690f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:15.200141 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a91e0-3f0a-4d22-a3b1-d6b2ed690f58 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:15.200429 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a91e0-3f0a-4d22-a3b1-d6b2ed690f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:15.200710 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a91e0-3f0a-4d22-a3b1-d6b2ed690f58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:15.201650 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99a91e0-3f0a-4d22-a3b1-d6b2ed690f58 None admin] RBAC: Adding query filter params (role.id=df30c7d320e947eaabb270652cb4afa3, scope.project.id=04d09da302f44fefbe5b4502010ec050, user.id=a40da6e662ee449cb5dd0f29b2025ab5) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:15.202113 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99a91e0-3f0a-4d22-a3b1-d6b2ed690f58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:15.203741 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99a91e0-3f0a-4d22-a3b1-d6b2ed690f58 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:15.217607 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 65/134] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:02:14 2022] GET /identity/v3/role_assignments?user.id=a40da6e662ee449cb5dd0f29b2025ab5&scope.project.id=04d09da302f44fefbe5b4502010ec050&role.id=df30c7d320e947eaabb270652cb4afa3 => generated 259 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:15.218509 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99a91e0-3f0a-4d22-a3b1-d6b2ed690f58 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:15.305350 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:15.327200 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef193607-0c39-4b48-aae8-378544881761 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:15.371615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5692c41-9069-4962-8751-d50b21fdbcba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:15.372597 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5692c41-9069-4962-8751-d50b21fdbcba None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:15.373028 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5692c41-9069-4962-8751-d50b21fdbcba None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:15.373321 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5692c41-9069-4962-8751-d50b21fdbcba None admin] PATH_INFO: `/v3/projects/b2f7d83a41384ce5bfae26b46ae88e1f` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:15.374813 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5692c41-9069-4962-8751-d50b21fdbcba None admin] RBAC: Authorizing `identity:get_project(project_id=b2f7d83a41384ce5bfae26b46ae88e1f)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:15.376521 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5692c41-9069-4962-8751-d50b21fdbcba None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:15.378653 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 70/135] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:15 2022] GET /identity/v3/projects/b2f7d83a41384ce5bfae26b46ae88e1f => generated 312 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:15.385674 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5692c41-9069-4962-8751-d50b21fdbcba None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:15.423178 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99a91e0-3f0a-4d22-a3b1-d6b2ed690f58 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:15.471382 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca3acc2-809f-4021-82a8-108d733653f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:15.472314 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3acc2-809f-4021-82a8-108d733653f8 None admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:15.472598 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3acc2-809f-4021-82a8-108d733653f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:15.472871 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3acc2-809f-4021-82a8-108d733653f8 None admin] PATH_INFO: `/v3/projects/04d09da302f44fefbe5b4502010ec050/users/ed277620276442838d05c774bf48c920/roles/5834784bf54d4806b88e32e0fb8e18fe` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:15.479309 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:15.482446 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca3acc2-809f-4021-82a8-108d733653f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=04d09da302f44fefbe5b4502010ec050, user_id=ed277620276442838d05c774bf48c920, role_id=5834784bf54d4806b88e32e0fb8e18fe)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:15.484021 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca3acc2-809f-4021-82a8-108d733653f8 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:15.508287 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 66/136] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 27 21:02:15 2022] PUT /identity/v3/projects/04d09da302f44fefbe5b4502010ec050/users/ed277620276442838d05c774bf48c920/roles/5834784bf54d4806b88e32e0fb8e18fe => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:02:15.565921 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5692c41-9069-4962-8751-d50b21fdbcba None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:15.612597 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe19bf35-2a89-4399-9d20-5ea79e70951e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:15.613638 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19bf35-2a89-4399-9d20-5ea79e70951e None admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:15.613972 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19bf35-2a89-4399-9d20-5ea79e70951e None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:15.614302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19bf35-2a89-4399-9d20-5ea79e70951e None admin] PATH_INFO: `/v3/projects/b2f7d83a41384ce5bfae26b46ae88e1f/users/ed277620276442838d05c774bf48c920/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:15.618475 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe19bf35-2a89-4399-9d20-5ea79e70951e None admin] RBAC: Authorizing `identity:create_grant(project_id=b2f7d83a41384ce5bfae26b46ae88e1f, user_id=ed277620276442838d05c774bf48c920, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:15.620208 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe19bf35-2a89-4399-9d20-5ea79e70951e None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:15.638872 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 71/137] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 27 21:02:15 2022] PUT /identity/v3/projects/b2f7d83a41384ce5bfae26b46ae88e1f/users/ed277620276442838d05c774bf48c920/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:02:15.908391 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae97d3-75f7-466c-acd9-3081620328ee None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:15.908949 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae97d3-75f7-466c-acd9-3081620328ee None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:15.909032 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae97d3-75f7-466c-acd9-3081620328ee None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:15.910560 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 67/138] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:15.917268 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da34cb7-6a6e-4305-ae2a-98de527b1bfe None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:15.917560 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da34cb7-6a6e-4305-ae2a-98de527b1bfe None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:15.917838 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da34cb7-6a6e-4305-ae2a-98de527b1bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:15.940384 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-7da34cb7-6a6e-4305-ae2a-98de527b1bfe None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:15.941632 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-7da34cb7-6a6e-4305-ae2a-98de527b1bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:16.014919 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db090960-d279-476f-be07-843ed38251ff None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:16.015141 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db090960-d279-476f-be07-843ed38251ff None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:16.015367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db090960-d279-476f-be07-843ed38251ff None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:16.019802 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 68/139] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:02:16.027971 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 72/140] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:16.035510 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632283a-b3b9-48c9-ba85-89fecbf2b450 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:16.035798 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632283a-b3b9-48c9-ba85-89fecbf2b450 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:16.036076 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632283a-b3b9-48c9-ba85-89fecbf2b450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:16.058663 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da34cb7-6a6e-4305-ae2a-98de527b1bfe None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:16.060057 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-a632283a-b3b9-48c9-ba85-89fecbf2b450 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:16.061301 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a632283a-b3b9-48c9-ba85-89fecbf2b450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:16.171429 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:16.210328 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 69/141] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:16.211780 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a864ecd-a28d-4d1e-a69b-b8d2b320aaba None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:16.212074 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a864ecd-a28d-4d1e-a69b-b8d2b320aaba None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:16.212353 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a864ecd-a28d-4d1e-a69b-b8d2b320aaba None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:16.213647 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 70/142] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:16.220334 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26664d1-2f92-41af-97a0-d77681e2e6d1 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:16.220620 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26664d1-2f92-41af-97a0-d77681e2e6d1 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:16.220902 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26664d1-2f92-41af-97a0-d77681e2e6d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:16.244742 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da34cb7-6a6e-4305-ae2a-98de527b1bfe None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:16.251422 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-b26664d1-2f92-41af-97a0-d77681e2e6d1 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:16.252630 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-b26664d1-2f92-41af-97a0-d77681e2e6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:16.281902 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:16.325082 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59413714-f14d-43f0-a850-3eefbbe4e233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:16.326089 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59413714-f14d-43f0-a850-3eefbbe4e233 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:16.326371 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59413714-f14d-43f0-a850-3eefbbe4e233 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:16.326643 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59413714-f14d-43f0-a850-3eefbbe4e233 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:16.327857 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59413714-f14d-43f0-a850-3eefbbe4e233 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:16.329369 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59413714-f14d-43f0-a850-3eefbbe4e233 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:16.331195 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 73/143] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:02:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:16.331908 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59413714-f14d-43f0-a850-3eefbbe4e233 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:16.334033 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 71/144] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:16.337897 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26664d1-2f92-41af-97a0-d77681e2e6d1 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:16.466316 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26664d1-2f92-41af-97a0-d77681e2e6d1 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:16.502747 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caed4ad-b0fa-44a0-9756-dc48e0e65085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:16.503628 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caed4ad-b0fa-44a0-9756-dc48e0e65085 None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:16.503917 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caed4ad-b0fa-44a0-9756-dc48e0e65085 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:16.504200 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caed4ad-b0fa-44a0-9756-dc48e0e65085 None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:16.504948 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caed4ad-b0fa-44a0-9756-dc48e0e65085 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:16.506452 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caed4ad-b0fa-44a0-9756-dc48e0e65085 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:16.507779 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59413714-f14d-43f0-a850-3eefbbe4e233 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:16.532838 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:16.539992 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 72/145] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Jun 27 21:02:16 2022] POST /identity/v3/users => generated 283 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:02:16.541846 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caed4ad-b0fa-44a0-9756-dc48e0e65085 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:16.580702 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa41e2-ccf5-49ff-b5ba-fd270838c0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:16.582132 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa41e2-ccf5-49ff-b5ba-fd270838c0ad None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:16.582344 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:16.582512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa41e2-ccf5-49ff-b5ba-fd270838c0ad None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:16.582665 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa41e2-ccf5-49ff-b5ba-fd270838c0ad None admin] PATH_INFO: `/v3/roles/member` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:16.606681 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aa41e2-ccf5-49ff-b5ba-fd270838c0ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:16.624838 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-92aa41e2-ccf5-49ff-b5ba-fd270838c0ad None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:16.624838 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:16.624838 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:16.624838 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:16.624838 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:16.624838 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:16.624838 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:16.624838 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:16.624838 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:16.624838 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:16.624838 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:16.624838 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:16.624838 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:16.624838 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:16.624838 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:16.624838 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:16.627092 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:16.627092 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:16.627092 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:16.627092 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:16.627092 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:16.627092 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:16.627092 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:16.627092 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:16.627092 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:16.627092 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:16.627092 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:16.627092 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:16.627092 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:16.627092 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:16.627092 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:16.627092 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:16.629137 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:16.629137 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:16.629137 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:16.629137 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:16.629137 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:16.629137 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:16.629137 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:16.629137 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:16.629137 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:16.629137 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:16.629137 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:16.629137 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:02:16.629137 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 74/146] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:16.636491 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa41e2-ccf5-49ff-b5ba-fd270838c0ad None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:16.758506 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caed4ad-b0fa-44a0-9756-dc48e0e65085 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:16.811794 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9358ca-8ae7-4e0b-a5bb-b0126a86a49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:16.812796 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9358ca-8ae7-4e0b-a5bb-b0126a86a49f None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:16.813125 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9358ca-8ae7-4e0b-a5bb-b0126a86a49f None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:16.813450 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9358ca-8ae7-4e0b-a5bb-b0126a86a49f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:16.824990 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9358ca-8ae7-4e0b-a5bb-b0126a86a49f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:16.833052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-5d9358ca-8ae7-4e0b-a5bb-b0126a86a49f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:02:16.833052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:16.833052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:16.833052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:16.833052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:16.833052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:16.833052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:16.833052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:16.833052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:16.833052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:16.833052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:16.833052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:16.833052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:16.833052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:16.833052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:16.833052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:16.835514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:16.835514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:16.835514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:16.835514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:16.835514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:16.835514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:16.835514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:16.835514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:16.835514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:16.835514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:16.835514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:16.835514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:16.835514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:16.835514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:16.835514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:16.835514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:16.837537 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:16.837537 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:16.837537 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:16.837537 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:16.837537 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:16.837537 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:16.837537 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:16.837537 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:16.837537 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:16.837537 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:16.837537 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:02:16.837537 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:02:16.837537 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 73/147] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 27 21:02:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:16.842187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9358ca-8ae7-4e0b-a5bb-b0126a86a49f None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:16.859773 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa41e2-ccf5-49ff-b5ba-fd270838c0ad None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:16.906999 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab05dc-af96-41c0-85bc-e9d850279eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:16.907694 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab05dc-af96-41c0-85bc-e9d850279eef None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:16.907982 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab05dc-af96-41c0-85bc-e9d850279eef None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:16.908256 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab05dc-af96-41c0-85bc-e9d850279eef None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:16.909062 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ab05dc-af96-41c0-85bc-e9d850279eef None admin] RBAC: Adding query filter params (name=member) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:16.909376 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ab05dc-af96-41c0-85bc-e9d850279eef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:16.910867 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ab05dc-af96-41c0-85bc-e9d850279eef None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:16.934228 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 75/148] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 27 21:02:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:16.940994 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab05dc-af96-41c0-85bc-e9d850279eef None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:17.009725 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:17.017413 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:17.057658 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9358ca-8ae7-4e0b-a5bb-b0126a86a49f None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:17.104742 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b964d7c-548d-4c47-ac33-e0d3ebb0a08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:17.105812 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b964d7c-548d-4c47-ac33-e0d3ebb0a08f None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:17.106316 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b964d7c-548d-4c47-ac33-e0d3ebb0a08f None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:17.106861 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b964d7c-548d-4c47-ac33-e0d3ebb0a08f None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:17.107799 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b964d7c-548d-4c47-ac33-e0d3ebb0a08f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:17.108302 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b964d7c-548d-4c47-ac33-e0d3ebb0a08f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:17.109739 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab05dc-af96-41c0-85bc-e9d850279eef None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:17.110105 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b964d7c-548d-4c47-ac33-e0d3ebb0a08f None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:17.126057 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 74/149] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:17.144063 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b964d7c-548d-4c47-ac33-e0d3ebb0a08f None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:17.164261 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c37dd-79c0-4d7b-8eb4-d4d5d89a49c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:17.165252 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c37dd-79c0-4d7b-8eb4-d4d5d89a49c6 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:17.165536 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c37dd-79c0-4d7b-8eb4-d4d5d89a49c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:17.165853 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c37dd-79c0-4d7b-8eb4-d4d5d89a49c6 None admin] PATH_INFO: `/v3/users/ed277620276442838d05c774bf48c920` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:17.167179 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3c37dd-79c0-4d7b-8eb4-d4d5d89a49c6 None admin] RBAC: Authorizing `identity:get_user(user_id=ed277620276442838d05c774bf48c920)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:17.169648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3c37dd-79c0-4d7b-8eb4-d4d5d89a49c6 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:17.172090 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 76/150] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:16 2022] GET /identity/v3/users/ed277620276442838d05c774bf48c920 => generated 275 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:17.177745 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c37dd-79c0-4d7b-8eb4-d4d5d89a49c6 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:17.318393 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b964d7c-548d-4c47-ac33-e0d3ebb0a08f None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:17.362222 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:17.367014 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c37dd-79c0-4d7b-8eb4-d4d5d89a49c6 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:17.373720 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af299594-c93f-4792-b872-5066c8f7f6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:17.374860 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af299594-c93f-4792-b872-5066c8f7f6da None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:17.375169 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af299594-c93f-4792-b872-5066c8f7f6da None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:17.375470 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af299594-c93f-4792-b872-5066c8f7f6da None admin] PATH_INFO: `/v3/users/a9e7736617b74291b419604a9ff9d7e1` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:17.381803 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af299594-c93f-4792-b872-5066c8f7f6da None admin] RBAC: Authorizing `identity:get_user(user_id=a9e7736617b74291b419604a9ff9d7e1)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:17.383398 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af299594-c93f-4792-b872-5066c8f7f6da None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:17.385462 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 75/151] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:17 2022] GET /identity/v3/users/a9e7736617b74291b419604a9ff9d7e1 => generated 247 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:17.414025 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af299594-c93f-4792-b872-5066c8f7f6da None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:17.418358 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c48f5-4b8b-4b42-943c-916c398109a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:17.419062 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c48f5-4b8b-4b42-943c-916c398109a6 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:17.419351 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c48f5-4b8b-4b42-943c-916c398109a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:17.419632 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c48f5-4b8b-4b42-943c-916c398109a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:17.429290 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:17.503154 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78c48f5-4b8b-4b42-943c-916c398109a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:17.504675 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78c48f5-4b8b-4b42-943c-916c398109a6 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:17.551220 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af299594-c93f-4792-b872-5066c8f7f6da None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:17.608624 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfce38a-a377-4025-929a-00ddc6959d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:17.609496 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfce38a-a377-4025-929a-00ddc6959d98 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:17.610189 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfce38a-a377-4025-929a-00ddc6959d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:17.610536 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfce38a-a377-4025-929a-00ddc6959d98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:17.638617 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:17.649335 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 77/152] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 21:02:17.657219 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c48f5-4b8b-4b42-943c-916c398109a6 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:17.746220 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfce38a-a377-4025-929a-00ddc6959d98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:17.747873 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfce38a-a377-4025-929a-00ddc6959d98 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:17.764727 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:17.823655 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c48f5-4b8b-4b42-943c-916c398109a6 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:17.866559 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cfa2d7-54d7-4270-aa58-aa157721676e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:17.867633 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfa2d7-54d7-4270-aa58-aa157721676e None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:17.868348 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfa2d7-54d7-4270-aa58-aa157721676e None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:17.868655 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfa2d7-54d7-4270-aa58-aa157721676e None admin] PATH_INFO: `/v3/projects/04d09da302f44fefbe5b4502010ec050` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:17.869970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cfa2d7-54d7-4270-aa58-aa157721676e None admin] RBAC: Authorizing `identity:get_project(project_id=04d09da302f44fefbe5b4502010ec050)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:17.872340 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cfa2d7-54d7-4270-aa58-aa157721676e None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:17.879026 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 78/153] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:17 2022] GET /identity/v3/projects/04d09da302f44fefbe5b4502010ec050 => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:17.887418 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cfa2d7-54d7-4270-aa58-aa157721676e None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:17.973584 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:17.975967 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 76/154] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:02:17.998498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfce38a-a377-4025-929a-00ddc6959d98 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:18.015100 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cfa2d7-54d7-4270-aa58-aa157721676e None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:18.075203 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9578861-ef7e-4388-9d31-18c84869a80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:18.075883 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9578861-ef7e-4388-9d31-18c84869a80d None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:18.076162 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9578861-ef7e-4388-9d31-18c84869a80d None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:18.076445 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9578861-ef7e-4388-9d31-18c84869a80d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:18.077346 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9578861-ef7e-4388-9d31-18c84869a80d None admin] RBAC: Adding query filter params (role.id=e4414bde4df44de8aacb36c585a93e07, scope.project.id=04d09da302f44fefbe5b4502010ec050, user.id=ed277620276442838d05c774bf48c920) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:18.077659 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9578861-ef7e-4388-9d31-18c84869a80d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:18.079131 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9578861-ef7e-4388-9d31-18c84869a80d None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:18.096548 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 79/155] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:02:17 2022] GET /identity/v3/role_assignments?user.id=ed277620276442838d05c774bf48c920&scope.project.id=04d09da302f44fefbe5b4502010ec050&role.id=e4414bde4df44de8aacb36c585a93e07 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:18.172286 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfce38a-a377-4025-929a-00ddc6959d98 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:18.183870 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:18.219571 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17409d89-fec7-4a18-a005-a990d6821e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:18.220455 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17409d89-fec7-4a18-a005-a990d6821e1d None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:18.220734 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17409d89-fec7-4a18-a005-a990d6821e1d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:18.221029 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17409d89-fec7-4a18-a005-a990d6821e1d None admin] PATH_INFO: `/v3/projects/04d09da302f44fefbe5b4502010ec050` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:18.222294 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17409d89-fec7-4a18-a005-a990d6821e1d None admin] RBAC: Authorizing `identity:get_project(project_id=04d09da302f44fefbe5b4502010ec050)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:18.223750 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17409d89-fec7-4a18-a005-a990d6821e1d None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:18.225798 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 77/156] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:17 2022] GET /identity/v3/projects/04d09da302f44fefbe5b4502010ec050 => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:18.232091 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9578861-ef7e-4388-9d31-18c84869a80d None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:18.271500 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fea244-7109-42f6-b8da-a2ff7b30fa89 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:18.271863 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fea244-7109-42f6-b8da-a2ff7b30fa89 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:18.272078 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fea244-7109-42f6-b8da-a2ff7b30fa89 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:18.274457 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 78/157] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:18.289068 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8d8e8-ef65-42eb-b405-7b58126454e1 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:18.289406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8d8e8-ef65-42eb-b405-7b58126454e1 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:18.289627 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8d8e8-ef65-42eb-b405-7b58126454e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:18.322018 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-74c8d8e8-ef65-42eb-b405-7b58126454e1 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:18.323420 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-74c8d8e8-ef65-42eb-b405-7b58126454e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:18.426968 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9578861-ef7e-4388-9d31-18c84869a80d None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:18.463131 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 79/158] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:02:18.501440 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c8d8e8-ef65-42eb-b405-7b58126454e1 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:18.505781 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:18.516871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bafe130-1571-47f9-836c-a9cabf5cc704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:18.521118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bafe130-1571-47f9-836c-a9cabf5cc704 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:18.521118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bafe130-1571-47f9-836c-a9cabf5cc704 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:18.521118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bafe130-1571-47f9-836c-a9cabf5cc704 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:18.521118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bafe130-1571-47f9-836c-a9cabf5cc704 None admin] RBAC: Adding query filter params (role.id=d336dd7b746c4488b33289c00302e4d5, scope.project.id=04d09da302f44fefbe5b4502010ec050, user.id=a9e7736617b74291b419604a9ff9d7e1) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:18.521118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bafe130-1571-47f9-836c-a9cabf5cc704 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:18.521118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bafe130-1571-47f9-836c-a9cabf5cc704 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:18.535446 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 80/159] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:02:18 2022] GET /identity/v3/role_assignments?user.id=a9e7736617b74291b419604a9ff9d7e1&scope.project.id=04d09da302f44fefbe5b4502010ec050&role.id=d336dd7b746c4488b33289c00302e4d5 => generated 608 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:18.668426 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c8d8e8-ef65-42eb-b405-7b58126454e1 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:18.677704 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:18.714367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f7e4f-61a7-4da5-8296-01f536fc41c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:18.715542 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f7e4f-61a7-4da5-8296-01f536fc41c2 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:18.715857 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f7e4f-61a7-4da5-8296-01f536fc41c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:18.716132 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f7e4f-61a7-4da5-8296-01f536fc41c2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:18.732893 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544f7e4f-61a7-4da5-8296-01f536fc41c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:18.745156 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-544f7e4f-61a7-4da5-8296-01f536fc41c2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 21:02:18.745156 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:18.745156 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:18.745156 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:18.745156 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:18.745156 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:18.745156 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:18.745156 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:18.745156 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:18.745156 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:18.745156 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:18.745156 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:18.745156 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:18.745156 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:18.745156 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:18.745156 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:18.746332 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:18.746332 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:18.746332 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:18.746332 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:18.746332 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:18.746332 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:18.746332 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:18.746332 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:18.746332 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:18.746332 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:18.746332 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:18.746332 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:18.746332 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:18.746332 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:18.746332 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:18.746332 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:18.747115 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:18.747115 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:18.747115 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:18.747115 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:18.747115 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:18.747115 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:18.747115 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:18.747115 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:18.747115 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:18.747115 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:18.747115 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 21:02:18.747115 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:02:18.747115 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 80/160] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:02:18.753254 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bafe130-1571-47f9-836c-a9cabf5cc704 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:18.871606 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bafe130-1571-47f9-836c-a9cabf5cc704 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:18.913096 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0e4b65-0a93-4493-81df-67023f96a1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:18.913787 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e4b65-0a93-4493-81df-67023f96a1a1 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:18.914066 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e4b65-0a93-4493-81df-67023f96a1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:18.914335 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e4b65-0a93-4493-81df-67023f96a1a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:18.915102 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0e4b65-0a93-4493-81df-67023f96a1a1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:18.915403 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0e4b65-0a93-4493-81df-67023f96a1a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:18.916798 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0e4b65-0a93-4493-81df-67023f96a1a1 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:18.926500 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 81/161] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 27 21:02:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:18.932491 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f7e4f-61a7-4da5-8296-01f536fc41c2 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:19.086706 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f7e4f-61a7-4da5-8296-01f536fc41c2 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:19.135847 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9359c180-e754-426f-bc54-516808f4c61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:19.136799 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359c180-e754-426f-bc54-516808f4c61c None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:19.137111 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359c180-e754-426f-bc54-516808f4c61c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:19.137421 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359c180-e754-426f-bc54-516808f4c61c None admin] PATH_INFO: `/v3/users/a40da6e662ee449cb5dd0f29b2025ab5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:19.141682 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9359c180-e754-426f-bc54-516808f4c61c None admin] RBAC: Authorizing `identity:get_user(user_id=a40da6e662ee449cb5dd0f29b2025ab5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:19.143162 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9359c180-e754-426f-bc54-516808f4c61c None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:19.145050 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 81/162] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:18 2022] GET /identity/v3/users/a40da6e662ee449cb5dd0f29b2025ab5 => generated 289 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:19.150544 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0e4b65-0a93-4493-81df-67023f96a1a1 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:19.196350 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1faf4e4-e609-4f71-9358-d2b345e3bee6 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:19.196883 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1faf4e4-e609-4f71-9358-d2b345e3bee6 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:19.196883 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1faf4e4-e609-4f71-9358-d2b345e3bee6 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:19.198378 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 82/163] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:02:19.217895 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c85384d-f63e-4c96-95da-dc62d0382490 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:19.218225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c85384d-f63e-4c96-95da-dc62d0382490 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:19.218451 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c85384d-f63e-4c96-95da-dc62d0382490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:19.231368 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:19.246351 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-7c85384d-f63e-4c96-95da-dc62d0382490 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:19.247961 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-7c85384d-f63e-4c96-95da-dc62d0382490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:19.283531 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:19.319636 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 83/164] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:02:19.338091 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0e4b65-0a93-4493-81df-67023f96a1a1 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:19.370970 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c85384d-f63e-4c96-95da-dc62d0382490 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:19.401324 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96ddbe-2e68-4b85-b405-bdeffcd2fbef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:19.402980 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96ddbe-2e68-4b85-b405-bdeffcd2fbef None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:19.406638 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96ddbe-2e68-4b85-b405-bdeffcd2fbef None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:19.407224 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96ddbe-2e68-4b85-b405-bdeffcd2fbef None admin] PATH_INFO: `/v3/projects/04d09da302f44fefbe5b4502010ec050` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:19.409151 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad96ddbe-2e68-4b85-b405-bdeffcd2fbef None admin] RBAC: Authorizing `identity:get_project(project_id=04d09da302f44fefbe5b4502010ec050)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:19.412052 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad96ddbe-2e68-4b85-b405-bdeffcd2fbef None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:19.416809 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 82/165] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:19 2022] GET /identity/v3/projects/04d09da302f44fefbe5b4502010ec050 => generated 298 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:19.426026 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96ddbe-2e68-4b85-b405-bdeffcd2fbef None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:19.588273 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c85384d-f63e-4c96-95da-dc62d0382490 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:19.644632 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96ddbe-2e68-4b85-b405-bdeffcd2fbef None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:19.654800 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76003ffe-4390-4099-9904-33de469d7ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:19.655628 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76003ffe-4390-4099-9904-33de469d7ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:19.655901 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76003ffe-4390-4099-9904-33de469d7ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:19.656170 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76003ffe-4390-4099-9904-33de469d7ad7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:19.675414 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76003ffe-4390-4099-9904-33de469d7ad7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:19.683385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-76003ffe-4390-4099-9904-33de469d7ad7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:02:19.683385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:19.683385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:19.683385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:19.683385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:19.683385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:19.683385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:19.683385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:19.683385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:19.683385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:19.683385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:19.683385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:19.683385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:19.683385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:19.683385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:19.683385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:19.684406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:19.684406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:19.684406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:19.684406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:19.684406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:19.684406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:19.684406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:19.684406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:19.684406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:19.684406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:19.684406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:19.684406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:19.684406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:19.684406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:19.684406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:19.684406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:19.685376 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:19.685376 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:19.685376 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:19.685376 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:19.685376 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:19.685376 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:19.685376 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:19.685376 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:19.685376 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:19.685376 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:19.685376 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:02:19.685376 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:02:19.685376 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 84/166] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 27 21:02:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:19.697308 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:19.698265 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31cefd-35e4-4943-b0f1-ac1bf569570d None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:19.698570 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31cefd-35e4-4943-b0f1-ac1bf569570d None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:19.698818 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31cefd-35e4-4943-b0f1-ac1bf569570d None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:19.700482 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 85/167] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:19 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:19.701939 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31cefd-35e4-4943-b0f1-ac1bf569570d None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:19.721121 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4998a4-ab44-419a-a071-74451eaca7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:19.722406 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4998a4-ab44-419a-a071-74451eaca7fc None admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:19.724480 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4998a4-ab44-419a-a071-74451eaca7fc None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:19.725057 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4998a4-ab44-419a-a071-74451eaca7fc None admin] PATH_INFO: `/v3/projects/04d09da302f44fefbe5b4502010ec050/users/a40da6e662ee449cb5dd0f29b2025ab5/roles/df30c7d320e947eaabb270652cb4afa3` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:19.729766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4998a4-ab44-419a-a071-74451eaca7fc None admin] RBAC: Authorizing `identity:create_grant(project_id=04d09da302f44fefbe5b4502010ec050, user_id=a40da6e662ee449cb5dd0f29b2025ab5, role_id=df30c7d320e947eaabb270652cb4afa3)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:19.731452 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4998a4-ab44-419a-a071-74451eaca7fc None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:19.741310 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:19.765299 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 83/168] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 27 21:02:19 2022] PUT /identity/v3/projects/04d09da302f44fefbe5b4502010ec050/users/a40da6e662ee449cb5dd0f29b2025ab5/roles/df30c7d320e947eaabb270652cb4afa3 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:02:19.767512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9b36e-d16a-401e-a3fc-0203df9b516b None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:19.767912 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9b36e-d16a-401e-a3fc-0203df9b516b None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:19.768124 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9b36e-d16a-401e-a3fc-0203df9b516b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:19.805155 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-fbf9b36e-d16a-401e-a3fc-0203df9b516b None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:19.806584 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-fbf9b36e-d16a-401e-a3fc-0203df9b516b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:19.836826 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31cefd-35e4-4943-b0f1-ac1bf569570d None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:19.877597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc73dfe0-5248-41f0-ab99-071cf51d63d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:19.884953 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc73dfe0-5248-41f0-ab99-071cf51d63d5 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:19.885921 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc73dfe0-5248-41f0-ab99-071cf51d63d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:19.886408 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc73dfe0-5248-41f0-ab99-071cf51d63d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:19.887751 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc73dfe0-5248-41f0-ab99-071cf51d63d5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:19.888291 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc73dfe0-5248-41f0-ab99-071cf51d63d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:19.890122 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc73dfe0-5248-41f0-ab99-071cf51d63d5 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:19.904826 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 86/169] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:02:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:19.910590 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc73dfe0-5248-41f0-ab99-071cf51d63d5 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:19.971108 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 84/170] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:20.008394 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9b36e-d16a-401e-a3fc-0203df9b516b None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:20.097351 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:20.115149 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc73dfe0-5248-41f0-ab99-071cf51d63d5 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:20.187598 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802421c-9e17-4ece-8eed-12fa52a0d2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:20.188537 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802421c-9e17-4ece-8eed-12fa52a0d2ff None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:20.188827 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802421c-9e17-4ece-8eed-12fa52a0d2ff None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:20.189146 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802421c-9e17-4ece-8eed-12fa52a0d2ff None admin] PATH_INFO: `/v3/users/ed277620276442838d05c774bf48c920` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:20.191482 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2802421c-9e17-4ece-8eed-12fa52a0d2ff None admin] RBAC: Authorizing `identity:get_user(user_id=ed277620276442838d05c774bf48c920)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:20.193064 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2802421c-9e17-4ece-8eed-12fa52a0d2ff None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:20.196174 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 87/171] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:19 2022] GET /identity/v3/users/ed277620276442838d05c774bf48c920 => generated 275 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:20.203014 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802421c-9e17-4ece-8eed-12fa52a0d2ff None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:20.229438 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9b36e-d16a-401e-a3fc-0203df9b516b None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:20.253205 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:20.276033 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc911c-7b41-433f-855c-34b79f245e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:20.277164 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc911c-7b41-433f-855c-34b79f245e94 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:20.277680 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc911c-7b41-433f-855c-34b79f245e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:20.278124 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc911c-7b41-433f-855c-34b79f245e94 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:20.292014 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddc911c-7b41-433f-855c-34b79f245e94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:20.301163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-dddc911c-7b41-433f-855c-34b79f245e94 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:20.301163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:20.301163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:20.301163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:20.301163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:20.301163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:20.301163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:20.301163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:20.301163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:20.301163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:20.301163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:20.301163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:20.301163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:20.301163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:20.301163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:20.301163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:20.302079 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:20.302079 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:20.302079 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:20.302079 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:20.302079 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:20.302079 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:20.302079 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:20.302079 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:20.302079 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:20.302079 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:20.302079 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:20.302079 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:20.302079 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:20.302079 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:20.302079 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:20.302079 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:20.302875 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:20.302875 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:20.302875 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:20.302875 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:20.302875 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:20.302875 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:20.302875 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:20.302875 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:20.302875 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:20.302875 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:20.302875 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:20.302875 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:02:20.303482 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 85/172] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:02:20.309843 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc911c-7b41-433f-855c-34b79f245e94 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:20.454939 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802421c-9e17-4ece-8eed-12fa52a0d2ff None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:20.481494 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc911c-7b41-433f-855c-34b79f245e94 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:20.529529 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c981cef4-2053-4796-b601-63fb88c79b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:20.530236 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c981cef4-2053-4796-b601-63fb88c79b3c None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:20.530539 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c981cef4-2053-4796-b601-63fb88c79b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:20.530834 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c981cef4-2053-4796-b601-63fb88c79b3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:20.561021 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c071f44f-257e-405d-823c-a6133f861e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:20.561813 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c071f44f-257e-405d-823c-a6133f861e64 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:20.562107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c071f44f-257e-405d-823c-a6133f861e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:20.562402 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c071f44f-257e-405d-823c-a6133f861e64 None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:20.564502 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c071f44f-257e-405d-823c-a6133f861e64 None admin] RBAC: Adding query filter params (name=member) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:20.564831 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c071f44f-257e-405d-823c-a6133f861e64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:20.566377 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c071f44f-257e-405d-823c-a6133f861e64 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:20.588958 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:20.592020 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 86/173] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 27 21:02:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:20.594833 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c071f44f-257e-405d-823c-a6133f861e64 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:20.667331 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:20.668104 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c981cef4-2053-4796-b601-63fb88c79b3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:20.670246 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c981cef4-2053-4796-b601-63fb88c79b3c None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:20.736493 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c071f44f-257e-405d-823c-a6133f861e64 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:20.775276 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1c9900-4cd0-42ab-a2e6-573b8125400b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:20.776256 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c9900-4cd0-42ab-a2e6-573b8125400b None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:20.776537 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c9900-4cd0-42ab-a2e6-573b8125400b None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:20.776809 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c9900-4cd0-42ab-a2e6-573b8125400b None admin] PATH_INFO: `/v3/users/ed277620276442838d05c774bf48c920` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:20.778237 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1c9900-4cd0-42ab-a2e6-573b8125400b None admin] RBAC: Authorizing `identity:get_user(user_id=ed277620276442838d05c774bf48c920)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:20.779857 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1c9900-4cd0-42ab-a2e6-573b8125400b None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:20.781543 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 87/174] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:20 2022] GET /identity/v3/users/ed277620276442838d05c774bf48c920 => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:20.782734 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c6bb4-3f23-47a3-920b-405ee00d34e5 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:20.783010 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c6bb4-3f23-47a3-920b-405ee00d34e5 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:20.783289 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c6bb4-3f23-47a3-920b-405ee00d34e5 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:20.789139 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 88/175] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:20.790464 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41038e9-10fc-4418-ba44-2be8bc0361f7 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:20.790749 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41038e9-10fc-4418-ba44-2be8bc0361f7 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:20.791026 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41038e9-10fc-4418-ba44-2be8bc0361f7 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:20.796751 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 89/176] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:20.797619 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41038e9-10fc-4418-ba44-2be8bc0361f7 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:20.818022 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 88/177] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 615 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:02:20.819521 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbee25a-b1f3-426a-b518-83e1c48db8e6 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:20.819813 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbee25a-b1f3-426a-b518-83e1c48db8e6 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:20.820092 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbee25a-b1f3-426a-b518-83e1c48db8e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:20.841684 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-7cbee25a-b1f3-426a-b518-83e1c48db8e6 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:20.843440 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-7cbee25a-b1f3-426a-b518-83e1c48db8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:20.909834 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:20.910028 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41038e9-10fc-4418-ba44-2be8bc0361f7 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:20.918640 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:20.939316 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 89/178] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:20.940429 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d3648-6ce7-4bd9-a5bc-1f31b374de91 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:20.940709 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d3648-6ce7-4bd9-a5bc-1f31b374de91 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:20.949145 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d3648-6ce7-4bd9-a5bc-1f31b374de91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:20.955713 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fec4c7-9590-48fb-a07d-42b15d979a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:20.956432 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fec4c7-9590-48fb-a07d-42b15d979a74 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:20.956709 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fec4c7-9590-48fb-a07d-42b15d979a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:20.957012 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fec4c7-9590-48fb-a07d-42b15d979a74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:20.981588 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-787d3648-6ce7-4bd9-a5bc-1f31b374de91 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:20.982766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-787d3648-6ce7-4bd9-a5bc-1f31b374de91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:21.043636 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fec4c7-9590-48fb-a07d-42b15d979a74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:21.045417 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fec4c7-9590-48fb-a07d-42b15d979a74 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:21.058022 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 90/179] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:21.059147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d3648-6ce7-4bd9-a5bc-1f31b374de91 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:21.167551 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d3648-6ce7-4bd9-a5bc-1f31b374de91 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:21.173510 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 90/180] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:02:21.178307 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be824d-f9f5-40db-99e4-cb45937714e4 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:21.178589 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be824d-f9f5-40db-99e4-cb45937714e4 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:21.178865 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be824d-f9f5-40db-99e4-cb45937714e4 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:21.180175 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 91/181] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:21.185365 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be824d-f9f5-40db-99e4-cb45937714e4 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:21.186273 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:21.194806 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:21.211158 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf84ac3-bce2-4fb8-862f-1f83b2601745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:21.212640 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf84ac3-bce2-4fb8-862f-1f83b2601745 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:21.213109 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf84ac3-bce2-4fb8-862f-1f83b2601745 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:21.213390 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf84ac3-bce2-4fb8-862f-1f83b2601745 None admin] PATH_INFO: `/v3/projects/04d09da302f44fefbe5b4502010ec050` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:21.214610 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf84ac3-bce2-4fb8-862f-1f83b2601745 None admin] RBAC: Authorizing `identity:get_project(project_id=04d09da302f44fefbe5b4502010ec050)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:21.217060 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf84ac3-bce2-4fb8-862f-1f83b2601745 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:21.219053 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 91/182] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:21 2022] GET /identity/v3/projects/04d09da302f44fefbe5b4502010ec050 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:21.219661 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf84ac3-bce2-4fb8-862f-1f83b2601745 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:21.306067 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be824d-f9f5-40db-99e4-cb45937714e4 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:21.329538 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf84ac3-bce2-4fb8-862f-1f83b2601745 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:21.342781 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2037ee2-e692-4d07-a5c5-114285a93b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:21.343584 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2037ee2-e692-4d07-a5c5-114285a93b3e None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:21.343867 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2037ee2-e692-4d07-a5c5-114285a93b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:21.344130 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2037ee2-e692-4d07-a5c5-114285a93b3e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:21.345319 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2037ee2-e692-4d07-a5c5-114285a93b3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:21.346767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2037ee2-e692-4d07-a5c5-114285a93b3e None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:21.348690 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 92/183] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:02:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:21.349617 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2037ee2-e692-4d07-a5c5-114285a93b3e None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:21.370927 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe1ed7-6106-4043-910b-09bda7b476a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:21.371827 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe1ed7-6106-4043-910b-09bda7b476a1 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:21.372108 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe1ed7-6106-4043-910b-09bda7b476a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:21.372381 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe1ed7-6106-4043-910b-09bda7b476a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:21.385266 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe1ed7-6106-4043-910b-09bda7b476a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:21.393421 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-aefe1ed7-6106-4043-910b-09bda7b476a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:02:21.393421 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:21.393421 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:21.393421 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:21.393421 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:21.393421 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:21.393421 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:21.393421 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:21.393421 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:21.393421 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:21.393421 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:21.393421 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:21.393421 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:21.393421 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:21.393421 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:21.393421 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:21.394417 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:21.394417 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:21.394417 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:21.394417 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:21.394417 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:21.394417 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:21.394417 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:21.394417 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:21.394417 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:21.394417 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:21.394417 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:21.394417 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:21.394417 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:21.394417 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:21.394417 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:21.394417 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:21.395204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:21.395204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:21.395204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:21.395204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:21.395204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:21.395204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:21.395204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:21.395204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:21.395204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:21.395204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:21.395204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:02:21.395204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:02:21.395204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 92/184] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 27 21:02:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:21.396373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6356b268-ecd9-466e-bcf9-bbf5fe2a665c None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:21.396471 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6356b268-ecd9-466e-bcf9-bbf5fe2a665c None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:21.396971 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6356b268-ecd9-466e-bcf9-bbf5fe2a665c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:21.414435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-6356b268-ecd9-466e-bcf9-bbf5fe2a665c None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:21.415641 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-6356b268-ecd9-466e-bcf9-bbf5fe2a665c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:21.439092 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:21.452007 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:21.468649 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2037ee2-e692-4d07-a5c5-114285a93b3e None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:21.486163 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 93/185] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:21.486894 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6356b268-ecd9-466e-bcf9-bbf5fe2a665c None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:21.512059 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa1565-f96a-427d-8e61-3b428b7b5d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:21.513208 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa1565-f96a-427d-8e61-3b428b7b5d42 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:21.513642 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa1565-f96a-427d-8e61-3b428b7b5d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:21.513912 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa1565-f96a-427d-8e61-3b428b7b5d42 None admin] PATH_INFO: `/v3/projects/b2f7d83a41384ce5bfae26b46ae88e1f` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:21.515122 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa1565-f96a-427d-8e61-3b428b7b5d42 None admin] RBAC: Authorizing `identity:get_project(project_id=b2f7d83a41384ce5bfae26b46ae88e1f)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:21.516574 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa1565-f96a-427d-8e61-3b428b7b5d42 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:21.518328 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 93/186] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:21 2022] GET /identity/v3/projects/b2f7d83a41384ce5bfae26b46ae88e1f => generated 312 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:21.519074 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa1565-f96a-427d-8e61-3b428b7b5d42 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:21.607815 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6356b268-ecd9-466e-bcf9-bbf5fe2a665c None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:21.643230 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644e666-23ed-462e-b122-c048a52a0c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:21.643894 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644e666-23ed-462e-b122-c048a52a0c06 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:21.644177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644e666-23ed-462e-b122-c048a52a0c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:21.644446 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644e666-23ed-462e-b122-c048a52a0c06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:21.644759 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa1565-f96a-427d-8e61-3b428b7b5d42 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:21.645315 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8644e666-23ed-462e-b122-c048a52a0c06 None admin] RBAC: Adding query filter params (role.id=5834784bf54d4806b88e32e0fb8e18fe, scope.project.id=04d09da302f44fefbe5b4502010ec050, user.id=ed277620276442838d05c774bf48c920) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:21.645627 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8644e666-23ed-462e-b122-c048a52a0c06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:21.647092 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8644e666-23ed-462e-b122-c048a52a0c06 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:21.653324 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 94/187] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:02:21 2022] GET /identity/v3/role_assignments?user.id=ed277620276442838d05c774bf48c920&scope.project.id=04d09da302f44fefbe5b4502010ec050&role.id=5834784bf54d4806b88e32e0fb8e18fe => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:21.654134 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644e666-23ed-462e-b122-c048a52a0c06 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:21.718939 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cacf53a-0137-49fc-bef2-e2ad59bb669f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:21.720841 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cacf53a-0137-49fc-bef2-e2ad59bb669f None admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:21.722899 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:21.723218 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cacf53a-0137-49fc-bef2-e2ad59bb669f None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:21.724303 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cacf53a-0137-49fc-bef2-e2ad59bb669f None admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:21.726766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cacf53a-0137-49fc-bef2-e2ad59bb669f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:21.729988 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cacf53a-0137-49fc-bef2-e2ad59bb669f None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:21.763296 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:21.779499 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 94/188] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Jun 27 21:02:21 2022] POST /identity/v3/users => generated 297 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:02:21.782799 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cacf53a-0137-49fc-bef2-e2ad59bb669f None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:21.859212 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644e666-23ed-462e-b122-c048a52a0c06 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:21.910241 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a0a447-0a23-4766-bc3c-cea24ccaa635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:21.910936 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a0a447-0a23-4766-bc3c-cea24ccaa635 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:21.911217 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a0a447-0a23-4766-bc3c-cea24ccaa635 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:21.911488 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a0a447-0a23-4766-bc3c-cea24ccaa635 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:21.912265 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a0a447-0a23-4766-bc3c-cea24ccaa635 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:21.912570 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a0a447-0a23-4766-bc3c-cea24ccaa635 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:21.914028 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a0a447-0a23-4766-bc3c-cea24ccaa635 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:21.934947 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 95/189] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:21.937250 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a0a447-0a23-4766-bc3c-cea24ccaa635 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:21.999551 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cacf53a-0137-49fc-bef2-e2ad59bb669f None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:22.056785 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b9680-1b23-4754-bbbb-556bd7ff64ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:22.057670 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b9680-1b23-4754-bbbb-556bd7ff64ff None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:22.057948 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b9680-1b23-4754-bbbb-556bd7ff64ff None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:22.058246 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b9680-1b23-4754-bbbb-556bd7ff64ff None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:22.072082 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b9680-1b23-4754-bbbb-556bd7ff64ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:22.077702 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:22.082532 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-d79b9680-1b23-4754-bbbb-556bd7ff64ff None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:02:22.082532 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:22.082532 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:22.082532 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:22.082532 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:22.082532 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:22.082532 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:22.082532 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:22.082532 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:22.082532 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:22.082532 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:22.082532 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:22.082532 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:22.082532 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:22.082532 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:22.082532 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:22.083764 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:22.083764 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:22.083764 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:22.083764 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:22.083764 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:22.083764 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:22.083764 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:22.083764 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:22.083764 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:22.083764 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:22.083764 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:22.083764 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:22.083764 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:22.083764 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:22.083764 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:22.083764 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:22.084558 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:22.084558 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:22.084558 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:22.084558 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:22.084558 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:22.084558 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:22.084558 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:22.084558 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:22.084558 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:22.084558 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:22.084558 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:02:22.084558 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:02:22.085704 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a0a447-0a23-4766-bc3c-cea24ccaa635 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:22.085778 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 95/190] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 27 21:02:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:22.086820 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b9680-1b23-4754-bbbb-556bd7ff64ff None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:22.137647 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27844e03-f7bd-4deb-94b8-a7cf0d7a9c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:22.138233 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27844e03-f7bd-4deb-94b8-a7cf0d7a9c08 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:22.138508 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27844e03-f7bd-4deb-94b8-a7cf0d7a9c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:22.138776 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27844e03-f7bd-4deb-94b8-a7cf0d7a9c08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:22.139651 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27844e03-f7bd-4deb-94b8-a7cf0d7a9c08 None admin] RBAC: Adding query filter params (role.id=e4414bde4df44de8aacb36c585a93e07, scope.project.id=b2f7d83a41384ce5bfae26b46ae88e1f, user.id=ed277620276442838d05c774bf48c920) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:22.139959 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27844e03-f7bd-4deb-94b8-a7cf0d7a9c08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:22.141470 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27844e03-f7bd-4deb-94b8-a7cf0d7a9c08 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:22.153259 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 96/191] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:02:21 2022] GET /identity/v3/role_assignments?user.id=ed277620276442838d05c774bf48c920&scope.project.id=b2f7d83a41384ce5bfae26b46ae88e1f&role.id=e4414bde4df44de8aacb36c585a93e07 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:22.153937 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27844e03-f7bd-4deb-94b8-a7cf0d7a9c08 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:22.178541 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:22.246772 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b9680-1b23-4754-bbbb-556bd7ff64ff None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:22.288274 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425191e-15ff-4ad9-a7e6-497d9fd81da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:22.289728 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425191e-15ff-4ad9-a7e6-497d9fd81da4 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:22.290242 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425191e-15ff-4ad9-a7e6-497d9fd81da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:22.290746 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425191e-15ff-4ad9-a7e6-497d9fd81da4 None admin] PATH_INFO: `/v3/users/c73d62abc85e489b924d33d2c053ad21` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:22.306935 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27844e03-f7bd-4deb-94b8-a7cf0d7a9c08 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:22.308055 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a425191e-15ff-4ad9-a7e6-497d9fd81da4 None admin] RBAC: Authorizing `identity:get_user(user_id=c73d62abc85e489b924d33d2c053ad21)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:22.310348 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a425191e-15ff-4ad9-a7e6-497d9fd81da4 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:22.312488 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 96/192] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:22 2022] GET /identity/v3/users/c73d62abc85e489b924d33d2c053ad21 => generated 283 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:22.323489 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425191e-15ff-4ad9-a7e6-497d9fd81da4 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:22.351695 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4b98c-4282-40d6-ad87-cd51f03886b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:22.352862 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4b98c-4282-40d6-ad87-cd51f03886b2 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:22.353097 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4b98c-4282-40d6-ad87-cd51f03886b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:22.353336 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4b98c-4282-40d6-ad87-cd51f03886b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:22.354187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e4b98c-4282-40d6-ad87-cd51f03886b2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:22.354505 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e4b98c-4282-40d6-ad87-cd51f03886b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:22.356891 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e4b98c-4282-40d6-ad87-cd51f03886b2 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:22.374168 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 97/193] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:02:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:22.380102 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4b98c-4282-40d6-ad87-cd51f03886b2 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:22.442412 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:22.450783 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425191e-15ff-4ad9-a7e6-497d9fd81da4 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:22.470259 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:22.496270 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adcea2a-ad64-4152-9757-61b7dfc97218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:22.497000 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcea2a-ad64-4152-9757-61b7dfc97218 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:22.497290 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcea2a-ad64-4152-9757-61b7dfc97218 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:22.497622 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcea2a-ad64-4152-9757-61b7dfc97218 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:22.590352 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adcea2a-ad64-4152-9757-61b7dfc97218 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:22.591855 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adcea2a-ad64-4152-9757-61b7dfc97218 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:22.598919 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4b98c-4282-40d6-ad87-cd51f03886b2 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:22.636850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa2a085-a897-4163-b0da-60082b48a5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:22.637817 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2a085-a897-4163-b0da-60082b48a5d2 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:22.638098 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2a085-a897-4163-b0da-60082b48a5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:22.638373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa2a085-a897-4163-b0da-60082b48a5d2 None admin] PATH_INFO: `/v3/users/c73d62abc85e489b924d33d2c053ad21` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:22.639763 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa2a085-a897-4163-b0da-60082b48a5d2 None admin] RBAC: Authorizing `identity:get_user(user_id=c73d62abc85e489b924d33d2c053ad21)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:22.641395 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa2a085-a897-4163-b0da-60082b48a5d2 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:22.646856 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 98/194] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:22 2022] GET /identity/v3/users/c73d62abc85e489b924d33d2c053ad21 => generated 283 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:22.646856 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349d043-1e7c-4198-8ce9-eddd33cabf70 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:22.646856 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349d043-1e7c-4198-8ce9-eddd33cabf70 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:22.646856 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349d043-1e7c-4198-8ce9-eddd33cabf70 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:22.647219 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 99/195] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:22.652538 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4349d043-1e7c-4198-8ce9-eddd33cabf70 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:22.714600 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 97/196] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:02:22.716427 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c15265a-b7b8-4e9c-9c6f-d721f839e7c7 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:22.716892 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c15265a-b7b8-4e9c-9c6f-d721f839e7c7 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:22.717366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c15265a-b7b8-4e9c-9c6f-d721f839e7c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:22.735124 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-7c15265a-b7b8-4e9c-9c6f-d721f839e7c7 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:22.736693 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-7c15265a-b7b8-4e9c-9c6f-d721f839e7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:22.748843 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:22.770719 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:22.791355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4349d043-1e7c-4198-8ce9-eddd33cabf70 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:22.817572 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 98/197] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:22.819331 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c15265a-b7b8-4e9c-9c6f-d721f839e7c7 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:22.838531 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae480b64-a53a-4699-ac54-b3b12c881570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:22.839191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae480b64-a53a-4699-ac54-b3b12c881570 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:22.839518 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae480b64-a53a-4699-ac54-b3b12c881570 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:22.840146 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae480b64-a53a-4699-ac54-b3b12c881570 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:22.916654 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae480b64-a53a-4699-ac54-b3b12c881570 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:22.918952 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae480b64-a53a-4699-ac54-b3b12c881570 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:22.937805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c15265a-b7b8-4e9c-9c6f-d721f839e7c7 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:22.974260 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b8267e-2d9a-4bb6-b136-214b78083648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:22.975129 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8267e-2d9a-4bb6-b136-214b78083648 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:22.975412 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8267e-2d9a-4bb6-b136-214b78083648 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:22.975681 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8267e-2d9a-4bb6-b136-214b78083648 None admin] PATH_INFO: `/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:22.976949 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b8267e-2d9a-4bb6-b136-214b78083648 None admin] RBAC: Authorizing `identity:get_project(project_id=8fc48df8fb1f4de59d32fefd09a8ecae)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:22.978932 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b8267e-2d9a-4bb6-b136-214b78083648 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:22.980721 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 99/198] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:22 2022] GET /identity/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:22.981560 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b8267e-2d9a-4bb6-b136-214b78083648 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:22.998787 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:23.023078 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:23.033858 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 100/199] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:02:23.034863 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae480b64-a53a-4699-ac54-b3b12c881570 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:23.100767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b8267e-2d9a-4bb6-b136-214b78083648 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:23.139305 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a882f10-50d1-4f30-947c-3ff12b309bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:23.140369 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a882f10-50d1-4f30-947c-3ff12b309bfc None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:23.140791 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a882f10-50d1-4f30-947c-3ff12b309bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:23.141237 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a882f10-50d1-4f30-947c-3ff12b309bfc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:23.142182 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae480b64-a53a-4699-ac54-b3b12c881570 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:23.152767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a882f10-50d1-4f30-947c-3ff12b309bfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:23.161243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-0a882f10-50d1-4f30-947c-3ff12b309bfc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 21:02:23.161243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:23.161243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:23.161243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:23.161243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:23.161243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:23.161243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:23.161243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:23.161243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:23.161243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:23.161243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:23.161243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:23.161243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:23.161243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:23.161243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:23.161243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:23.162136 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:23.162136 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:23.162136 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:23.162136 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:23.162136 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:23.162136 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:23.162136 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:23.162136 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:23.162136 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:23.162136 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:23.162136 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:23.162136 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:23.162136 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:23.162136 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:23.162136 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:23.162136 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:23.162937 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:23.162937 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:23.162937 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:23.162937 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:23.162937 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:23.162937 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:23.162937 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:23.162937 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:23.162937 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:23.162937 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:23.162937 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 21:02:23.162937 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:02:23.165625 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 100/200] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:23.166753 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a882f10-50d1-4f30-947c-3ff12b309bfc None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:23.179044 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78842b0a-0297-4737-975e-694d22956c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:23.179827 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78842b0a-0297-4737-975e-694d22956c4d None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:23.180180 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78842b0a-0297-4737-975e-694d22956c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:23.180558 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78842b0a-0297-4737-975e-694d22956c4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:23.181713 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78842b0a-0297-4737-975e-694d22956c4d None admin] RBAC: Adding query filter params (role.id=d336dd7b746c4488b33289c00302e4d5, scope.project.id=8fc48df8fb1f4de59d32fefd09a8ecae, user.id=c73d62abc85e489b924d33d2c053ad21) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:23.181834 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78842b0a-0297-4737-975e-694d22956c4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:23.183378 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78842b0a-0297-4737-975e-694d22956c4d None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:23.190107 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 101/201] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:02:23 2022] GET /identity/v3/role_assignments?user.id=c73d62abc85e489b924d33d2c053ad21&scope.project.id=8fc48df8fb1f4de59d32fefd09a8ecae&role.id=d336dd7b746c4488b33289c00302e4d5 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:23.191105 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78842b0a-0297-4737-975e-694d22956c4d None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:23.258212 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:23.259561 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:23.289933 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a882f10-50d1-4f30-947c-3ff12b309bfc None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:23.329878 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b60f22-616a-4efc-a653-69c34ccc6087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:23.331159 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b60f22-616a-4efc-a653-69c34ccc6087 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:23.331704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b60f22-616a-4efc-a653-69c34ccc6087 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:23.332176 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b60f22-616a-4efc-a653-69c34ccc6087 None admin] PATH_INFO: `/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:23.333806 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b60f22-616a-4efc-a653-69c34ccc6087 None admin] RBAC: Authorizing `identity:get_project(project_id=8fc48df8fb1f4de59d32fefd09a8ecae)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:23.335487 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b60f22-616a-4efc-a653-69c34ccc6087 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:23.337564 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 101/202] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:23 2022] GET /identity/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:23.343154 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78842b0a-0297-4737-975e-694d22956c4d None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:23.344793 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b60f22-616a-4efc-a653-69c34ccc6087 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:23.431080 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797eae3c-9b18-493c-8418-45c7e6157dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:23.432281 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797eae3c-9b18-493c-8418-45c7e6157dc8 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:23.432863 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797eae3c-9b18-493c-8418-45c7e6157dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:23.433383 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797eae3c-9b18-493c-8418-45c7e6157dc8 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:23.434514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797eae3c-9b18-493c-8418-45c7e6157dc8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:23.435047 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797eae3c-9b18-493c-8418-45c7e6157dc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:23.437613 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797eae3c-9b18-493c-8418-45c7e6157dc8 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:23.448816 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 102/203] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 27 21:02:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:23.455058 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797eae3c-9b18-493c-8418-45c7e6157dc8 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:23.507689 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b60f22-616a-4efc-a653-69c34ccc6087 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:23.543367 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a90c8-9ac8-457c-8e54-421413bb7fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:23.543942 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a90c8-9ac8-457c-8e54-421413bb7fed None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:23.544227 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a90c8-9ac8-457c-8e54-421413bb7fed None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:23.544503 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a90c8-9ac8-457c-8e54-421413bb7fed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:23.545398 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2a90c8-9ac8-457c-8e54-421413bb7fed None admin] RBAC: Adding query filter params (role.id=5834784bf54d4806b88e32e0fb8e18fe, scope.project.id=8fc48df8fb1f4de59d32fefd09a8ecae, user.id=c73d62abc85e489b924d33d2c053ad21) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:23.545713 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2a90c8-9ac8-457c-8e54-421413bb7fed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:23.547408 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2a90c8-9ac8-457c-8e54-421413bb7fed None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:23.554617 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 102/204] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:02:23 2022] GET /identity/v3/role_assignments?user.id=c73d62abc85e489b924d33d2c053ad21&scope.project.id=8fc48df8fb1f4de59d32fefd09a8ecae&role.id=5834784bf54d4806b88e32e0fb8e18fe => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:23.586279 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797eae3c-9b18-493c-8418-45c7e6157dc8 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:23.606978 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:23.626202 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a9f05-4e2b-4dd7-bfa5-0e748a626554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:23.627254 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a9f05-4e2b-4dd7-bfa5-0e748a626554 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:23.627550 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a9f05-4e2b-4dd7-bfa5-0e748a626554 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:23.627828 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a9f05-4e2b-4dd7-bfa5-0e748a626554 None admin] PATH_INFO: `/v3/users/a40da6e662ee449cb5dd0f29b2025ab5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:23.629222 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6a9f05-4e2b-4dd7-bfa5-0e748a626554 None admin] RBAC: Authorizing `identity:get_user(user_id=a40da6e662ee449cb5dd0f29b2025ab5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:23.630731 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6a9f05-4e2b-4dd7-bfa5-0e748a626554 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:23.632869 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 103/205] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:23 2022] GET /identity/v3/users/a40da6e662ee449cb5dd0f29b2025ab5 => generated 289 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:23.638139 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a90c8-9ac8-457c-8e54-421413bb7fed None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:23.650635 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:23.781312 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a90c8-9ac8-457c-8e54-421413bb7fed None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:23.819758 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd18f70-4e9f-49d4-8aeb-c5828d4962a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:23.820441 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd18f70-4e9f-49d4-8aeb-c5828d4962a9 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:23.820728 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd18f70-4e9f-49d4-8aeb-c5828d4962a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:23.820912 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd18f70-4e9f-49d4-8aeb-c5828d4962a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:23.900121 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd18f70-4e9f-49d4-8aeb-c5828d4962a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:23.903023 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd18f70-4e9f-49d4-8aeb-c5828d4962a9 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:23.918960 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:24.056227 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 103/206] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 21:02:24.061684 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a9f05-4e2b-4dd7-bfa5-0e748a626554 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:24.219180 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a9f05-4e2b-4dd7-bfa5-0e748a626554 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:24.288824 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443309e7-68c2-453e-bd18-59d5e7162943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:24.290115 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443309e7-68c2-453e-bd18-59d5e7162943 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:24.290591 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443309e7-68c2-453e-bd18-59d5e7162943 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:24.291081 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443309e7-68c2-453e-bd18-59d5e7162943 None admin] PATH_INFO: `/v3/projects/04d09da302f44fefbe5b4502010ec050` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:24.292528 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443309e7-68c2-453e-bd18-59d5e7162943 None admin] RBAC: Authorizing `identity:get_project(project_id=04d09da302f44fefbe5b4502010ec050)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:24.294470 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443309e7-68c2-453e-bd18-59d5e7162943 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:24.296731 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 104/207] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:24 2022] GET /identity/v3/projects/04d09da302f44fefbe5b4502010ec050 => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:24.302275 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd18f70-4e9f-49d4-8aeb-c5828d4962a9 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:24.414566 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd18f70-4e9f-49d4-8aeb-c5828d4962a9 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:24.434034 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:24.452971 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a751a-ea5e-4a13-82c0-ef409f2846ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:24.454061 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a751a-ea5e-4a13-82c0-ef409f2846ee None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:24.454345 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a751a-ea5e-4a13-82c0-ef409f2846ee None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:24.454619 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a751a-ea5e-4a13-82c0-ef409f2846ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:24.455472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458a751a-ea5e-4a13-82c0-ef409f2846ee None admin] RBAC: Adding query filter params (role.id=df30c7d320e947eaabb270652cb4afa3, scope.project.id=04d09da302f44fefbe5b4502010ec050, user.id=a40da6e662ee449cb5dd0f29b2025ab5) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:24.455787 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458a751a-ea5e-4a13-82c0-ef409f2846ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:24.457286 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458a751a-ea5e-4a13-82c0-ef409f2846ee None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:24.463871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 104/208] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:02:24 2022] GET /identity/v3/role_assignments?user.id=a40da6e662ee449cb5dd0f29b2025ab5&scope.project.id=04d09da302f44fefbe5b4502010ec050&role.id=df30c7d320e947eaabb270652cb4afa3 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:24.607619 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece2ac4-ca5b-477e-8e1d-6694b85b5e94 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:24.608582 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece2ac4-ca5b-477e-8e1d-6694b85b5e94 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:24.609082 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece2ac4-ca5b-477e-8e1d-6694b85b5e94 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:24.611159 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 105/209] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:24.619078 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de15bcc-0014-4a68-bf07-175fcb5baba1 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:24.619422 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de15bcc-0014-4a68-bf07-175fcb5baba1 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:24.619638 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de15bcc-0014-4a68-bf07-175fcb5baba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:24.650270 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-3de15bcc-0014-4a68-bf07-175fcb5baba1 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:24.651524 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-3de15bcc-0014-4a68-bf07-175fcb5baba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:24.720273 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427fe04d-c79c-4dfa-ab00-2e2b37dbebae None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:24.720654 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427fe04d-c79c-4dfa-ab00-2e2b37dbebae None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:24.720852 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427fe04d-c79c-4dfa-ab00-2e2b37dbebae None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:24.722512 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 106/210] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:24.729926 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b696f7b-5998-49bf-9243-939e8eb3305a None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:24.730208 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b696f7b-5998-49bf-9243-939e8eb3305a None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:24.730517 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b696f7b-5998-49bf-9243-939e8eb3305a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:24.743393 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 105/211] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:24.752916 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-2b696f7b-5998-49bf-9243-939e8eb3305a None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:24.754614 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-2b696f7b-5998-49bf-9243-939e8eb3305a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:24.772640 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de15bcc-0014-4a68-bf07-175fcb5baba1 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:24.822938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:24.849151 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 107/212] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:24.874443 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b696f7b-5998-49bf-9243-939e8eb3305a None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:24.902878 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:24.907485 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de15bcc-0014-4a68-bf07-175fcb5baba1 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:24.969465 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7752db2-c3d9-4be9-ad19-738da666341b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:24.970815 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7752db2-c3d9-4be9-ad19-738da666341b None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:24.971330 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7752db2-c3d9-4be9-ad19-738da666341b None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:24.971847 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7752db2-c3d9-4be9-ad19-738da666341b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:24.974589 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7752db2-c3d9-4be9-ad19-738da666341b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:24.977993 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7752db2-c3d9-4be9-ad19-738da666341b None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:24.985010 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 106/213] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:02:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:24.989840 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7752db2-c3d9-4be9-ad19-738da666341b None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:25.020306 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b696f7b-5998-49bf-9243-939e8eb3305a None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:25.066354 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9112f8d-7e4d-43db-ac91-0d3faf9a1789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:25.067663 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9112f8d-7e4d-43db-ac91-0d3faf9a1789 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:25.068328 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9112f8d-7e4d-43db-ac91-0d3faf9a1789 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:25.068634 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9112f8d-7e4d-43db-ac91-0d3faf9a1789 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:25.082571 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9112f8d-7e4d-43db-ac91-0d3faf9a1789 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:25.091727 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-e9112f8d-7e4d-43db-ac91-0d3faf9a1789 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:25.091727 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:25.091727 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:25.091727 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:25.091727 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:25.091727 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:25.091727 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:25.091727 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:25.091727 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:25.091727 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:25.091727 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:25.091727 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:25.091727 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:25.091727 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:25.091727 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:25.091727 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:25.093261 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:25.093261 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:25.093261 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:25.093261 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:25.093261 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:25.093261 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:25.093261 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:25.093261 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:25.093261 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:25.093261 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:25.093261 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:25.093261 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:25.093261 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:25.093261 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:25.093261 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:25.093261 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:25.094613 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:25.094613 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:25.094613 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:25.094613 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:25.094613 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:25.094613 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:25.094613 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:25.094613 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:25.094613 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:25.094613 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:25.094613 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:25.094613 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:02:25.094613 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 108/214] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:02:25.099896 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9112f8d-7e4d-43db-ac91-0d3faf9a1789 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:25.161689 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7752db2-c3d9-4be9-ad19-738da666341b None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:25.172574 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:25.201171 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6a709-cf1e-4dfa-93e2-dc9a8f156f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:25.202073 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6a709-cf1e-4dfa-93e2-dc9a8f156f17 None admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:25.202211 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6a709-cf1e-4dfa-93e2-dc9a8f156f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:25.202493 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6a709-cf1e-4dfa-93e2-dc9a8f156f17 None admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:25.203303 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb6a709-cf1e-4dfa-93e2-dc9a8f156f17 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:25.204758 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb6a709-cf1e-4dfa-93e2-dc9a8f156f17 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:25.236914 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 107/215] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Jun 27 21:02:24 2022] POST /identity/v3/users => generated 297 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:02:25.254363 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9112f8d-7e4d-43db-ac91-0d3faf9a1789 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:25.294150 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e953380-8bc3-44a5-b1ab-68a60e55a823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:25.295433 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e953380-8bc3-44a5-b1ab-68a60e55a823 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:25.295891 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e953380-8bc3-44a5-b1ab-68a60e55a823 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:25.296365 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e953380-8bc3-44a5-b1ab-68a60e55a823 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:25.297380 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e953380-8bc3-44a5-b1ab-68a60e55a823 None admin] RBAC: Adding query filter params (name=member) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:25.297884 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e953380-8bc3-44a5-b1ab-68a60e55a823 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:25.299579 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e953380-8bc3-44a5-b1ab-68a60e55a823 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:25.310306 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 109/216] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 27 21:02:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:25.316963 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6a709-cf1e-4dfa-93e2-dc9a8f156f17 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:25.348412 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:25.443028 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6a709-cf1e-4dfa-93e2-dc9a8f156f17 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:25.481753 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26add07f-9330-4e47-8e31-4d280f96917c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:25.482960 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26add07f-9330-4e47-8e31-4d280f96917c None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:25.483440 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26add07f-9330-4e47-8e31-4d280f96917c None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:25.483899 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26add07f-9330-4e47-8e31-4d280f96917c None admin] PATH_INFO: `/v3/users/5ebbceb49ff04c3f841e47c3aaa9d960` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:25.506746 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26add07f-9330-4e47-8e31-4d280f96917c None admin] RBAC: Authorizing `identity:get_user(user_id=5ebbceb49ff04c3f841e47c3aaa9d960)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:25.508877 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26add07f-9330-4e47-8e31-4d280f96917c None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:25.511435 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 108/217] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:25 2022] GET /identity/v3/users/5ebbceb49ff04c3f841e47c3aaa9d960 => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:25.517356 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e953380-8bc3-44a5-b1ab-68a60e55a823 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:25.637237 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e953380-8bc3-44a5-b1ab-68a60e55a823 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:25.646348 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:25.683159 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb86efe-eb8d-46ef-abc9-6d8caa030e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:25.684290 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb86efe-eb8d-46ef-abc9-6d8caa030e3a None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:25.684833 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb86efe-eb8d-46ef-abc9-6d8caa030e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:25.685418 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb86efe-eb8d-46ef-abc9-6d8caa030e3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:25.698110 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e41462-6d34-4169-a512-8c61df669415 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:25.698452 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e41462-6d34-4169-a512-8c61df669415 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:25.698643 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e41462-6d34-4169-a512-8c61df669415 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:25.700596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 109/218] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:25.706761 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed8ca3-1dc7-40d4-942b-dfa33693ae1c None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:25.707044 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed8ca3-1dc7-40d4-942b-dfa33693ae1c None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:25.707320 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed8ca3-1dc7-40d4-942b-dfa33693ae1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:25.728783 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-60ed8ca3-1dc7-40d4-942b-dfa33693ae1c None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:25.730017 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-60ed8ca3-1dc7-40d4-942b-dfa33693ae1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:25.792286 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb86efe-eb8d-46ef-abc9-6d8caa030e3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:25.794512 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb86efe-eb8d-46ef-abc9-6d8caa030e3a None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:25.807143 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 110/219] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:02:25.895608 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed8ca3-1dc7-40d4-942b-dfa33693ae1c None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:25.916909 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:26.004163 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 110/220] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:02:26.010223 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb86efe-eb8d-46ef-abc9-6d8caa030e3a None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:26.025165 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed8ca3-1dc7-40d4-942b-dfa33693ae1c None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:26.053383 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:26.057984 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d293035-7415-4319-adc7-1dc679d3dafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:26.059716 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d293035-7415-4319-adc7-1dc679d3dafe None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:26.060256 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d293035-7415-4319-adc7-1dc679d3dafe None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:26.060728 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d293035-7415-4319-adc7-1dc679d3dafe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:26.082825 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d293035-7415-4319-adc7-1dc679d3dafe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:26.095840 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-1d293035-7415-4319-adc7-1dc679d3dafe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:02:26.095840 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:26.095840 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:26.095840 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:26.095840 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:26.095840 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:26.095840 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:26.095840 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:26.095840 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:26.095840 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:26.095840 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:26.095840 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:26.095840 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:26.095840 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:26.095840 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:26.095840 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:26.096805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:26.096805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:26.096805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:26.096805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:26.096805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:26.096805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:26.096805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:26.096805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:26.096805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:26.096805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:26.096805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:26.096805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:26.096805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:26.096805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:26.096805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:26.096805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:26.097634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:26.097634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:26.097634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:26.097634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:26.097634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:26.097634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:26.097634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:26.097634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:26.097634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:26.097634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:26.097634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:02:26.097634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:02:26.105022 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 111/221] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 27 21:02:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:02:26.107663 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d293035-7415-4319-adc7-1dc679d3dafe None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:26.184861 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:26.244057 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d293035-7415-4319-adc7-1dc679d3dafe None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:26.298993 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35455b32-91ec-4576-b644-28f67f4fb3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:26.300582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35455b32-91ec-4576-b644-28f67f4fb3ef None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:26.301191 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35455b32-91ec-4576-b644-28f67f4fb3ef None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:26.301661 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35455b32-91ec-4576-b644-28f67f4fb3ef None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:26.302621 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35455b32-91ec-4576-b644-28f67f4fb3ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:26.303138 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35455b32-91ec-4576-b644-28f67f4fb3ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:26.304899 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35455b32-91ec-4576-b644-28f67f4fb3ef None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:26.316156 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb86efe-eb8d-46ef-abc9-6d8caa030e3a None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:26.321061 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 112/222] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:26.323507 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab6112-41c8-4209-aeee-16e10e1611c3 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:26.324229 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab6112-41c8-4209-aeee-16e10e1611c3 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:26.324809 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab6112-41c8-4209-aeee-16e10e1611c3 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:26.326455 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 113/223] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:26.329541 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab6112-41c8-4209-aeee-16e10e1611c3 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:26.387615 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaf7ab2-26da-4684-99c7-70f2b5ac73e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:26.388510 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf7ab2-26da-4684-99c7-70f2b5ac73e5 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:26.388818 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf7ab2-26da-4684-99c7-70f2b5ac73e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:26.389122 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf7ab2-26da-4684-99c7-70f2b5ac73e5 None admin] PATH_INFO: `/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:26.393860 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaf7ab2-26da-4684-99c7-70f2b5ac73e5 None admin] RBAC: Authorizing `identity:get_project(project_id=8fc48df8fb1f4de59d32fefd09a8ecae)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:26.395381 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaf7ab2-26da-4684-99c7-70f2b5ac73e5 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:26.397454 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 111/224] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:26 2022] GET /identity/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae => generated 302 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:26.398887 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76a00d-8259-4191-b1fd-41b6f05160f8 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:26.399273 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76a00d-8259-4191-b1fd-41b6f05160f8 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:26.399692 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76a00d-8259-4191-b1fd-41b6f05160f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:26.417984 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-8f76a00d-8259-4191-b1fd-41b6f05160f8 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:26.419526 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-8f76a00d-8259-4191-b1fd-41b6f05160f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:26.524950 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 112/225] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:26.524950 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f76a00d-8259-4191-b1fd-41b6f05160f8 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:26.534196 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:26.575720 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab6112-41c8-4209-aeee-16e10e1611c3 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:26.648621 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18816a-c666-4194-a4a8-bb2a2331ec07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:26.650072 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18816a-c666-4194-a4a8-bb2a2331ec07 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:26.655004 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18816a-c666-4194-a4a8-bb2a2331ec07 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:26.655571 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18816a-c666-4194-a4a8-bb2a2331ec07 None admin] PATH_INFO: `/v3/users/c73d62abc85e489b924d33d2c053ad21` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:26.657219 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18816a-c666-4194-a4a8-bb2a2331ec07 None admin] RBAC: Authorizing `identity:get_user(user_id=c73d62abc85e489b924d33d2c053ad21)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:26.658900 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18816a-c666-4194-a4a8-bb2a2331ec07 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:26.663904 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 114/226] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:26 2022] GET /identity/v3/users/c73d62abc85e489b924d33d2c053ad21 => generated 283 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:26.666488 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18816a-c666-4194-a4a8-bb2a2331ec07 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:26.677975 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:26.709373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f76a00d-8259-4191-b1fd-41b6f05160f8 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:26.758060 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9285fdca-3de5-4981-bfc2-cd45c4f9c515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:26.758703 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285fdca-3de5-4981-bfc2-cd45c4f9c515 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:26.758981 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285fdca-3de5-4981-bfc2-cd45c4f9c515 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:26.759254 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285fdca-3de5-4981-bfc2-cd45c4f9c515 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:26.760676 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9285fdca-3de5-4981-bfc2-cd45c4f9c515 None admin] RBAC: Adding query filter params (role.id=e4414bde4df44de8aacb36c585a93e07, scope.project.id=8fc48df8fb1f4de59d32fefd09a8ecae, user.id=5ebbceb49ff04c3f841e47c3aaa9d960) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:26.761034 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9285fdca-3de5-4981-bfc2-cd45c4f9c515 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:26.763225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9285fdca-3de5-4981-bfc2-cd45c4f9c515 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:26.771311 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 113/227] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:02:26 2022] GET /identity/v3/role_assignments?user.id=5ebbceb49ff04c3f841e47c3aaa9d960&scope.project.id=8fc48df8fb1f4de59d32fefd09a8ecae&role.id=e4414bde4df44de8aacb36c585a93e07 => generated 259 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:26.772343 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9285fdca-3de5-4981-bfc2-cd45c4f9c515 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:26.855754 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18816a-c666-4194-a4a8-bb2a2331ec07 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:26.869386 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:26.917626 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9285fdca-3de5-4981-bfc2-cd45c4f9c515 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:26.955226 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e07e32c-0823-4e82-8c65-cbdb135fae67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:26.956077 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07e32c-0823-4e82-8c65-cbdb135fae67 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:26.956363 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07e32c-0823-4e82-8c65-cbdb135fae67 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:26.956666 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07e32c-0823-4e82-8c65-cbdb135fae67 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:26.977647 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea80ce9-9fe1-4c2c-ac35-70681106cb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:26.979008 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea80ce9-9fe1-4c2c-ac35-70681106cb64 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:26.979488 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea80ce9-9fe1-4c2c-ac35-70681106cb64 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:26.979959 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea80ce9-9fe1-4c2c-ac35-70681106cb64 None admin] PATH_INFO: `/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:26.982331 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea80ce9-9fe1-4c2c-ac35-70681106cb64 None admin] RBAC: Authorizing `identity:get_project(project_id=8fc48df8fb1f4de59d32fefd09a8ecae)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:26.986018 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea80ce9-9fe1-4c2c-ac35-70681106cb64 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:26.989015 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 114/228] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:26 2022] GET /identity/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:26.990261 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e07e32c-0823-4e82-8c65-cbdb135fae67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:26.995364 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea80ce9-9fe1-4c2c-ac35-70681106cb64 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:27.008614 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-9e07e32c-0823-4e82-8c65-cbdb135fae67 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:02:27.008614 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:27.008614 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:27.008614 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:27.008614 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:27.008614 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:27.008614 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:27.008614 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:27.008614 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:27.008614 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:27.008614 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:27.008614 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:27.008614 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:27.008614 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:27.008614 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:27.008614 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:27.009664 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:27.009664 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:27.009664 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:27.009664 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:27.009664 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:27.009664 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:27.009664 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:27.009664 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:27.009664 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:27.009664 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:27.009664 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:27.009664 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:27.009664 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:27.009664 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:27.009664 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:27.009664 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:27.010454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:27.010454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:27.010454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:27.010454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:27.010454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:27.010454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:27.010454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:27.010454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:27.010454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:27.010454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:27.010454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:02:27.010454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:02:27.015514 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 115/229] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 27 21:02:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:27.038755 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e07e32c-0823-4e82-8c65-cbdb135fae67 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:27.148389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea80ce9-9fe1-4c2c-ac35-70681106cb64 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:27.191021 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b8dbc-0394-4885-bff6-ddfdd92b7846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:27.191931 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b8dbc-0394-4885-bff6-ddfdd92b7846 None admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:27.193028 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b8dbc-0394-4885-bff6-ddfdd92b7846 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:27.193312 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b8dbc-0394-4885-bff6-ddfdd92b7846 None admin] PATH_INFO: `/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae/users/c73d62abc85e489b924d33d2c053ad21/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:27.196947 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:27.197263 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66b8dbc-0394-4885-bff6-ddfdd92b7846 None admin] RBAC: Authorizing `identity:create_grant(project_id=8fc48df8fb1f4de59d32fefd09a8ecae, user_id=c73d62abc85e489b924d33d2c053ad21, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:27.200530 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66b8dbc-0394-4885-bff6-ddfdd92b7846 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:27.208068 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:27.227658 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 115/230] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 27 21:02:26 2022] PUT /identity/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae/users/c73d62abc85e489b924d33d2c053ad21/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:02:27.258202 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e07e32c-0823-4e82-8c65-cbdb135fae67 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:27.304628 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b9c9c-f74d-4d88-bb45-37fc7433ce4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:27.305616 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b9c9c-f74d-4d88-bb45-37fc7433ce4a None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:27.305903 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b9c9c-f74d-4d88-bb45-37fc7433ce4a None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:27.306215 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b9c9c-f74d-4d88-bb45-37fc7433ce4a None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:27.307006 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4b9c9c-f74d-4d88-bb45-37fc7433ce4a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:27.307325 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4b9c9c-f74d-4d88-bb45-37fc7433ce4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:27.309045 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4b9c9c-f74d-4d88-bb45-37fc7433ce4a None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:27.322039 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 116/231] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:02:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:27.330482 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b8dbc-0394-4885-bff6-ddfdd92b7846 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:27.478884 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b8dbc-0394-4885-bff6-ddfdd92b7846 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:27.516221 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c37803-f715-47da-8997-a4bad320a356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:27.517181 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c37803-f715-47da-8997-a4bad320a356 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:27.517461 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c37803-f715-47da-8997-a4bad320a356 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:27.517731 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c37803-f715-47da-8997-a4bad320a356 None admin] PATH_INFO: `/v3/users/c73d62abc85e489b924d33d2c053ad21` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:27.518995 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c37803-f715-47da-8997-a4bad320a356 None admin] RBAC: Authorizing `identity:get_user(user_id=c73d62abc85e489b924d33d2c053ad21)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:27.520498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c37803-f715-47da-8997-a4bad320a356 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:27.522785 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 116/232] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:27 2022] GET /identity/v3/users/c73d62abc85e489b924d33d2c053ad21 => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:27.530978 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b9c9c-f74d-4d88-bb45-37fc7433ce4a None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:27.658481 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b9c9c-f74d-4d88-bb45-37fc7433ce4a None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:27.696497 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:27.721390 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f597885e-c43d-4e86-a03a-e6d7bc19b16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:27.722560 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f597885e-c43d-4e86-a03a-e6d7bc19b16a None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:27.722837 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f597885e-c43d-4e86-a03a-e6d7bc19b16a None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:27.723109 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f597885e-c43d-4e86-a03a-e6d7bc19b16a None admin] PATH_INFO: `/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:27.724318 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f597885e-c43d-4e86-a03a-e6d7bc19b16a None admin] RBAC: Authorizing `identity:get_project(project_id=8fc48df8fb1f4de59d32fefd09a8ecae)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:27.725808 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f597885e-c43d-4e86-a03a-e6d7bc19b16a None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:27.728063 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 117/233] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:27 2022] GET /identity/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:27.734145 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c37803-f715-47da-8997-a4bad320a356 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:27.807970 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:27.855770 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c37803-f715-47da-8997-a4bad320a356 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:27.908356 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc5fbe1-0e30-4b03-80dd-55acfc70f7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:27.909290 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5fbe1-0e30-4b03-80dd-55acfc70f7cd None admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:27.909570 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5fbe1-0e30-4b03-80dd-55acfc70f7cd None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:27.909847 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5fbe1-0e30-4b03-80dd-55acfc70f7cd None admin] PATH_INFO: `/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae/users/c73d62abc85e489b924d33d2c053ad21/roles/5834784bf54d4806b88e32e0fb8e18fe` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:27.913244 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc5fbe1-0e30-4b03-80dd-55acfc70f7cd None admin] RBAC: Authorizing `identity:create_grant(project_id=8fc48df8fb1f4de59d32fefd09a8ecae, user_id=c73d62abc85e489b924d33d2c053ad21, role_id=5834784bf54d4806b88e32e0fb8e18fe)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:27.914710 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc5fbe1-0e30-4b03-80dd-55acfc70f7cd None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:27.937524 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 117/234] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 27 21:02:27 2022] PUT /identity/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae/users/c73d62abc85e489b924d33d2c053ad21/roles/5834784bf54d4806b88e32e0fb8e18fe => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:02:27.973856 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b581df7-eaa6-40bb-bf10-27601f7b66d5 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:27.974518 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b581df7-eaa6-40bb-bf10-27601f7b66d5 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:27.974518 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b581df7-eaa6-40bb-bf10-27601f7b66d5 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:27.975824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 118/235] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:27.989201 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d7a16-2e94-4288-8637-95e1f3a89182 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:27.989520 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d7a16-2e94-4288-8637-95e1f3a89182 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:27.989679 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d7a16-2e94-4288-8637-95e1f3a89182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:28.017246 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-411d7a16-2e94-4288-8637-95e1f3a89182 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:28.019348 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-411d7a16-2e94-4288-8637-95e1f3a89182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:28.053513 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790ae73-69f6-4cdd-b638-f1db416c581b None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:28.053841 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790ae73-69f6-4cdd-b638-f1db416c581b None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:28.054068 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790ae73-69f6-4cdd-b638-f1db416c581b None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:28.055713 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 119/236] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:02:28.064523 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a168c56-0579-4ac9-b24f-e840cf36c313 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:28.064840 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a168c56-0579-4ac9-b24f-e840cf36c313 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:28.065109 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a168c56-0579-4ac9-b24f-e840cf36c313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:28.089250 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-1a168c56-0579-4ac9-b24f-e840cf36c313 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:28.090988 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-1a168c56-0579-4ac9-b24f-e840cf36c313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:28.157266 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 118/237] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:28.186633 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d7a16-2e94-4288-8637-95e1f3a89182 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:28.220411 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 120/238] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:28.248638 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a168c56-0579-4ac9-b24f-e840cf36c313 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:28.253533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:28.305676 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d7a16-2e94-4288-8637-95e1f3a89182 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:28.368205 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63e124-54df-421a-b5a6-7ea5ac1da509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:28.369049 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63e124-54df-421a-b5a6-7ea5ac1da509 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:28.369332 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63e124-54df-421a-b5a6-7ea5ac1da509 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:28.369604 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63e124-54df-421a-b5a6-7ea5ac1da509 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:28.391043 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b63e124-54df-421a-b5a6-7ea5ac1da509 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:28.398864 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-2b63e124-54df-421a-b5a6-7ea5ac1da509 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 21:02:28.398864 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:28.398864 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:28.398864 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:28.398864 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:28.398864 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:28.398864 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:28.398864 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:28.398864 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:28.398864 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:28.398864 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:28.398864 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:28.398864 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:28.398864 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:28.398864 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:28.398864 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:28.399761 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:28.399761 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:28.399761 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:28.399761 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:28.399761 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:28.399761 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:28.399761 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:28.399761 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:28.399761 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:28.399761 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:28.399761 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:28.399761 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:28.399761 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:28.399761 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:28.399761 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:28.399761 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:28.400568 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:28.400568 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:28.400568 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:28.400568 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:28.400568 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:28.400568 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:28.400568 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:28.400568 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:28.400568 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:28.400568 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:28.400568 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 21:02:28.400568 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:02:28.404945 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 119/239] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:02:28.411435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63e124-54df-421a-b5a6-7ea5ac1da509 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:28.413895 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a168c56-0579-4ac9-b24f-e840cf36c313 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:28.456097 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:28.477987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca36108-8440-4878-bb4b-016f8ae6a327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:28.478802 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca36108-8440-4878-bb4b-016f8ae6a327 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:28.479080 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca36108-8440-4878-bb4b-016f8ae6a327 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:28.479355 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca36108-8440-4878-bb4b-016f8ae6a327 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:28.480676 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca36108-8440-4878-bb4b-016f8ae6a327 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:28.482163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca36108-8440-4878-bb4b-016f8ae6a327 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:28.491298 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 121/240] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:02:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:28.497011 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca36108-8440-4878-bb4b-016f8ae6a327 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:28.605963 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63e124-54df-421a-b5a6-7ea5ac1da509 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:28.634949 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:28.670972 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2679b3f-3e30-462d-a2b8-1a99e684d3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:28.671655 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2679b3f-3e30-462d-a2b8-1a99e684d3ba None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:28.671935 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2679b3f-3e30-462d-a2b8-1a99e684d3ba None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:28.672210 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2679b3f-3e30-462d-a2b8-1a99e684d3ba None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:28.672995 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2679b3f-3e30-462d-a2b8-1a99e684d3ba None admin] RBAC: Adding query filter params (name=reader) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:28.673306 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2679b3f-3e30-462d-a2b8-1a99e684d3ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:28.674709 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2679b3f-3e30-462d-a2b8-1a99e684d3ba None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:28.679730 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca36108-8440-4878-bb4b-016f8ae6a327 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:28.704666 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 120/241] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 27 21:02:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:28.705771 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2679b3f-3e30-462d-a2b8-1a99e684d3ba None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:28.742588 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee184a01-b0ce-4738-8c71-e4f9e3974166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:28.743683 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee184a01-b0ce-4738-8c71-e4f9e3974166 None admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:28.744163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee184a01-b0ce-4738-8c71-e4f9e3974166 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:28.744661 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee184a01-b0ce-4738-8c71-e4f9e3974166 None admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:28.745614 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee184a01-b0ce-4738-8c71-e4f9e3974166 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:28.747305 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee184a01-b0ce-4738-8c71-e4f9e3974166 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:28.771521 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 122/242] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Jun 27 21:02:28 2022] POST /identity/v3/users => generated 293 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:02:28.906296 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2679b3f-3e30-462d-a2b8-1a99e684d3ba None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:28.958718 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014cd7e6-d134-48c9-80f5-6c06af4172b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:28.959632 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014cd7e6-d134-48c9-80f5-6c06af4172b4 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:28.959915 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014cd7e6-d134-48c9-80f5-6c06af4172b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:28.960188 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014cd7e6-d134-48c9-80f5-6c06af4172b4 None admin] PATH_INFO: `/v3/users/0ed54725ccd84ea19e42fb7a87144676` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:28.978542 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014cd7e6-d134-48c9-80f5-6c06af4172b4 None admin] RBAC: Authorizing `identity:get_user(user_id=0ed54725ccd84ea19e42fb7a87144676)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:28.980084 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014cd7e6-d134-48c9-80f5-6c06af4172b4 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:28.982088 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 121/243] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:28 2022] GET /identity/v3/users/0ed54725ccd84ea19e42fb7a87144676 => generated 297 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:28.988897 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee184a01-b0ce-4738-8c71-e4f9e3974166 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:29.052860 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:29.114402 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee184a01-b0ce-4738-8c71-e4f9e3974166 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:29.150229 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd269d15-671c-46b6-9824-bc2b4447805e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:29.151196 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd269d15-671c-46b6-9824-bc2b4447805e None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:29.151829 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd269d15-671c-46b6-9824-bc2b4447805e None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:29.152361 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd269d15-671c-46b6-9824-bc2b4447805e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:29.231870 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd269d15-671c-46b6-9824-bc2b4447805e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:29.233678 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd269d15-671c-46b6-9824-bc2b4447805e None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:29.284764 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:29.318413 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ac7b7-9453-412d-94ad-9a2ee8acc375 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:29.318735 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ac7b7-9453-412d-94ad-9a2ee8acc375 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:29.318980 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ac7b7-9453-412d-94ad-9a2ee8acc375 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:29.325235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 122/244] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:02:29.328248 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2fe7d-de01-4bb3-989a-93e62a7179cd None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:29.328856 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2fe7d-de01-4bb3-989a-93e62a7179cd None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:29.328856 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d2fe7d-de01-4bb3-989a-93e62a7179cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:29.351412 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-d6d2fe7d-de01-4bb3-989a-93e62a7179cd None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:29.353041 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d6d2fe7d-de01-4bb3-989a-93e62a7179cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:29.353712 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 123/245] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:02:29.359522 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd269d15-671c-46b6-9824-bc2b4447805e None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:29.400736 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:29.489432 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 123/246] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:29.515519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d2fe7d-de01-4bb3-989a-93e62a7179cd None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:29.516742 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd269d15-671c-46b6-9824-bc2b4447805e None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:29.566083 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855ca4e-ed52-4097-bcad-618a89dc5c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:29.567607 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855ca4e-ed52-4097-bcad-618a89dc5c10 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:29.568376 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855ca4e-ed52-4097-bcad-618a89dc5c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:29.568796 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855ca4e-ed52-4097-bcad-618a89dc5c10 None admin] PATH_INFO: `/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:29.570359 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d855ca4e-ed52-4097-bcad-618a89dc5c10 None admin] RBAC: Authorizing `identity:get_project(project_id=8fc48df8fb1f4de59d32fefd09a8ecae)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:29.572143 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d855ca4e-ed52-4097-bcad-618a89dc5c10 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:29.575792 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 124/247] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:29 2022] GET /identity/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:29.581707 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855ca4e-ed52-4097-bcad-618a89dc5c10 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:29.603155 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:29.658961 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d2fe7d-de01-4bb3-989a-93e62a7179cd None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:29.704648 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de946b7e-d5b5-452d-bfdf-2967f5e9a252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:29.706056 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de946b7e-d5b5-452d-bfdf-2967f5e9a252 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:29.706686 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de946b7e-d5b5-452d-bfdf-2967f5e9a252 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:29.707298 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de946b7e-d5b5-452d-bfdf-2967f5e9a252 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:29.724969 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de946b7e-d5b5-452d-bfdf-2967f5e9a252 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:29.726850 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855ca4e-ed52-4097-bcad-618a89dc5c10 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:29.734830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-de946b7e-d5b5-452d-bfdf-2967f5e9a252 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:29.734830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:29.734830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:29.734830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:29.734830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:29.734830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:29.734830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:29.734830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:29.734830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:29.734830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:29.734830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:29.734830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:29.734830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:29.734830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:29.734830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:29.734830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:29.735921 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:29.735921 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:29.735921 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:29.735921 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:29.735921 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:29.735921 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:29.735921 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:29.735921 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:29.735921 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:29.735921 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:29.735921 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:29.735921 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:29.735921 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:29.735921 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:29.735921 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:29.735921 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:29.736836 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:29.736836 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:29.736836 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:29.736836 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:29.736836 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:29.736836 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:29.736836 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:29.736836 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:29.736836 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:29.736836 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:29.736836 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:29.736836 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:02:29.738972 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 124/248] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:02:29.745219 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de946b7e-d5b5-452d-bfdf-2967f5e9a252 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:29.771924 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f430ad74-4822-4cca-8bf2-7d0894763047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:29.772632 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430ad74-4822-4cca-8bf2-7d0894763047 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:29.776979 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430ad74-4822-4cca-8bf2-7d0894763047 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:29.776979 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430ad74-4822-4cca-8bf2-7d0894763047 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:29.776979 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f430ad74-4822-4cca-8bf2-7d0894763047 None admin] RBAC: Adding query filter params (role.id=df30c7d320e947eaabb270652cb4afa3, scope.project.id=8fc48df8fb1f4de59d32fefd09a8ecae, user.id=0ed54725ccd84ea19e42fb7a87144676) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:29.776979 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f430ad74-4822-4cca-8bf2-7d0894763047 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:29.776979 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f430ad74-4822-4cca-8bf2-7d0894763047 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:29.784431 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 125/249] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:02:29 2022] GET /identity/v3/role_assignments?user.id=0ed54725ccd84ea19e42fb7a87144676&scope.project.id=8fc48df8fb1f4de59d32fefd09a8ecae&role.id=df30c7d320e947eaabb270652cb4afa3 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:29.803108 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:29.899540 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de946b7e-d5b5-452d-bfdf-2967f5e9a252 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:29.923450 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bc096-a280-4667-ae67-d13340acc737 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:29.923793 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bc096-a280-4667-ae67-d13340acc737 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:29.923999 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4bc096-a280-4667-ae67-d13340acc737 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:29.925479 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 126/250] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:02:29.933313 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82829875-23ee-4a57-8b5e-fbe42cb169fe None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:29.933313 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82829875-23ee-4a57-8b5e-fbe42cb169fe None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:29.934475 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82829875-23ee-4a57-8b5e-fbe42cb169fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:29.955132 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30852395-62f4-4f4c-8dc9-062ed6f3e393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:29.955834 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30852395-62f4-4f4c-8dc9-062ed6f3e393 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:29.956085 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30852395-62f4-4f4c-8dc9-062ed6f3e393 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:29.956356 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30852395-62f4-4f4c-8dc9-062ed6f3e393 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:29.957143 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30852395-62f4-4f4c-8dc9-062ed6f3e393 None admin] RBAC: Adding query filter params (name=member) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:29.957447 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30852395-62f4-4f4c-8dc9-062ed6f3e393 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:29.958852 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30852395-62f4-4f4c-8dc9-062ed6f3e393 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:29.966801 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-82829875-23ee-4a57-8b5e-fbe42cb169fe None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:29.968107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-82829875-23ee-4a57-8b5e-fbe42cb169fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:29.976885 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 125/251] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 27 21:02:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:29.983386 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30852395-62f4-4f4c-8dc9-062ed6f3e393 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:30.081006 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 127/252] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:30.107627 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82829875-23ee-4a57-8b5e-fbe42cb169fe None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:30.116984 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:30.128373 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:30.137983 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30852395-62f4-4f4c-8dc9-062ed6f3e393 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:30.186380 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e32ad68-6c30-4dc2-a3e7-b004b9029cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:30.187867 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32ad68-6c30-4dc2-a3e7-b004b9029cc8 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:30.188365 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32ad68-6c30-4dc2-a3e7-b004b9029cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:30.188825 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32ad68-6c30-4dc2-a3e7-b004b9029cc8 None admin] PATH_INFO: `/v3/users/5ebbceb49ff04c3f841e47c3aaa9d960` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:30.194596 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e32ad68-6c30-4dc2-a3e7-b004b9029cc8 None admin] RBAC: Authorizing `identity:get_user(user_id=5ebbceb49ff04c3f841e47c3aaa9d960)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:30.196095 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e32ad68-6c30-4dc2-a3e7-b004b9029cc8 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:30.197909 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 126/253] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:29 2022] GET /identity/v3/users/5ebbceb49ff04c3f841e47c3aaa9d960 => generated 297 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:30.206458 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e32ad68-6c30-4dc2-a3e7-b004b9029cc8 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:30.291590 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82829875-23ee-4a57-8b5e-fbe42cb169fe None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:30.380374 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbdb15e-96c7-4a60-8e64-46ff0b0c8a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:30.381263 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbdb15e-96c7-4a60-8e64-46ff0b0c8a16 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:30.381544 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbdb15e-96c7-4a60-8e64-46ff0b0c8a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:30.381816 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbdb15e-96c7-4a60-8e64-46ff0b0c8a16 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:30.413912 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbdb15e-96c7-4a60-8e64-46ff0b0c8a16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:30.418168 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e32ad68-6c30-4dc2-a3e7-b004b9029cc8 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:30.430256 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-fdbdb15e-96c7-4a60-8e64-46ff0b0c8a16 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:02:30.430256 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:30.430256 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:30.430256 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:30.430256 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:30.430256 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:30.430256 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:30.430256 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:30.430256 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:30.430256 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:30.430256 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:30.430256 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:30.430256 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:30.430256 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:30.430256 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:30.430256 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:30.431178 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:30.431178 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:30.431178 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:30.431178 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:30.431178 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:30.431178 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:30.431178 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:30.431178 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:30.431178 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:30.431178 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:30.431178 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:30.431178 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:30.431178 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:30.431178 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:30.431178 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:30.431178 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:30.431993 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:30.431993 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:30.431993 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:30.431993 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:30.431993 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:30.431993 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:30.431993 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:30.431993 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:30.431993 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:30.431993 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:30.431993 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:02:30.431993 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:02:30.431993 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 128/254] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 27 21:02:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:02:30.450337 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbdb15e-96c7-4a60-8e64-46ff0b0c8a16 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:30.480520 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096839b2-ffe7-44f9-b000-1e37b752342d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:30.481378 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096839b2-ffe7-44f9-b000-1e37b752342d None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:30.481661 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096839b2-ffe7-44f9-b000-1e37b752342d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:30.481938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096839b2-ffe7-44f9-b000-1e37b752342d None admin] PATH_INFO: `/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:30.485804 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096839b2-ffe7-44f9-b000-1e37b752342d None admin] RBAC: Authorizing `identity:get_project(project_id=8fc48df8fb1f4de59d32fefd09a8ecae)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:30.487498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096839b2-ffe7-44f9-b000-1e37b752342d None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:30.489610 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 127/255] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:30 2022] GET /identity/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae => generated 302 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:30.496882 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096839b2-ffe7-44f9-b000-1e37b752342d None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:30.547960 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:30.642293 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:30.694010 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096839b2-ffe7-44f9-b000-1e37b752342d None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:30.733062 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbdb15e-96c7-4a60-8e64-46ff0b0c8a16 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:30.767164 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6aa3c2-1d94-4115-b1e8-54f11422a690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:30.768512 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6aa3c2-1d94-4115-b1e8-54f11422a690 None admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:30.769016 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6aa3c2-1d94-4115-b1e8-54f11422a690 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:30.769485 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6aa3c2-1d94-4115-b1e8-54f11422a690 None admin] PATH_INFO: `/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae/users/5ebbceb49ff04c3f841e47c3aaa9d960/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:30.774792 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6aa3c2-1d94-4115-b1e8-54f11422a690 None admin] RBAC: Authorizing `identity:create_grant(project_id=8fc48df8fb1f4de59d32fefd09a8ecae, user_id=5ebbceb49ff04c3f841e47c3aaa9d960, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:30.776737 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6aa3c2-1d94-4115-b1e8-54f11422a690 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:30.800133 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 128/256] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 27 21:02:30 2022] PUT /identity/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae/users/5ebbceb49ff04c3f841e47c3aaa9d960/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:02:30.814933 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faabf121-f78b-4972-95c1-afaf27aa6eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:30.815638 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faabf121-f78b-4972-95c1-afaf27aa6eac None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:30.815922 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faabf121-f78b-4972-95c1-afaf27aa6eac None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:30.816196 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faabf121-f78b-4972-95c1-afaf27aa6eac None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:30.816997 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faabf121-f78b-4972-95c1-afaf27aa6eac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:30.817311 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faabf121-f78b-4972-95c1-afaf27aa6eac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:30.818712 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faabf121-f78b-4972-95c1-afaf27aa6eac None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:30.836990 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 129/257] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:30.842877 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6aa3c2-1d94-4115-b1e8-54f11422a690 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:30.969485 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:30.994304 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6aa3c2-1d94-4115-b1e8-54f11422a690 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:31.030635 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57b22e-38c4-42d3-af38-54d4b06d9b38 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:31.030987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57b22e-38c4-42d3-af38-54d4b06d9b38 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:31.031192 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57b22e-38c4-42d3-af38-54d4b06d9b38 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:31.036951 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 130/258] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:31.039133 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e22d4-8171-4f8d-9ad9-ca7a4258e0d0 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:31.040322 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e22d4-8171-4f8d-9ad9-ca7a4258e0d0 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:31.040686 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e22d4-8171-4f8d-9ad9-ca7a4258e0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:31.041991 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f11404b-f5c6-418d-80f8-6105fb256b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:31.043236 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11404b-f5c6-418d-80f8-6105fb256b22 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:31.043720 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11404b-f5c6-418d-80f8-6105fb256b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:31.044200 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11404b-f5c6-418d-80f8-6105fb256b22 None admin] PATH_INFO: `/v3/users/c73d62abc85e489b924d33d2c053ad21` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:31.049759 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f11404b-f5c6-418d-80f8-6105fb256b22 None admin] RBAC: Authorizing `identity:get_user(user_id=c73d62abc85e489b924d33d2c053ad21)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:31.051970 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f11404b-f5c6-418d-80f8-6105fb256b22 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:31.054436 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 129/259] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:30 2022] GET /identity/v3/users/c73d62abc85e489b924d33d2c053ad21 => generated 283 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:31.060174 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f11404b-f5c6-418d-80f8-6105fb256b22 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:31.070940 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-6d2e22d4-8171-4f8d-9ad9-ca7a4258e0d0 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:31.072156 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-6d2e22d4-8171-4f8d-9ad9-ca7a4258e0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:31.181091 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f11404b-f5c6-418d-80f8-6105fb256b22 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:31.226267 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37c6e0-f6b9-4f90-9f5b-c6d61562a68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:31.226940 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37c6e0-f6b9-4f90-9f5b-c6d61562a68e None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:31.227610 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37c6e0-f6b9-4f90-9f5b-c6d61562a68e None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:31.227886 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37c6e0-f6b9-4f90-9f5b-c6d61562a68e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:31.260188 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:31.262971 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 131/260] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:31.298137 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2e22d4-8171-4f8d-9ad9-ca7a4258e0d0 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:31.334220 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d37c6e0-f6b9-4f90-9f5b-c6d61562a68e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:31.335644 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d37c6e0-f6b9-4f90-9f5b-c6d61562a68e None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:31.442255 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:31.515675 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 130/261] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 21:02:31.523390 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37c6e0-f6b9-4f90-9f5b-c6d61562a68e None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:31.552263 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2e22d4-8171-4f8d-9ad9-ca7a4258e0d0 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:31.645559 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:31.649948 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe2c83-56fe-40c4-a5e9-2028d123b6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:31.650787 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe2c83-56fe-40c4-a5e9-2028d123b6a2 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:31.651069 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe2c83-56fe-40c4-a5e9-2028d123b6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:31.651344 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe2c83-56fe-40c4-a5e9-2028d123b6a2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:31.676996 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abe2c83-56fe-40c4-a5e9-2028d123b6a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:31.691667 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-8abe2c83-56fe-40c4-a5e9-2028d123b6a2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:02:31.691667 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:31.691667 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:31.691667 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:31.691667 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:31.691667 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:31.691667 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:31.691667 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:31.691667 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:31.691667 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:31.691667 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:31.691667 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:31.691667 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:31.691667 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:31.691667 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:31.691667 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:31.693235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:31.693235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:31.693235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:31.693235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:31.693235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:31.693235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:31.693235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:31.693235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:31.693235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:31.693235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:31.693235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:31.693235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:31.693235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:31.693235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:31.693235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:31.693235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:31.694039 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:31.694039 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:31.694039 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:31.694039 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:31.694039 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:31.694039 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:31.694039 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:31.694039 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:31.694039 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:31.694039 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:31.694039 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:02:31.694039 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:02:31.694039 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 132/262] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 27 21:02:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 396 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:31.701308 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe2c83-56fe-40c4-a5e9-2028d123b6a2 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:31.733223 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37c6e0-f6b9-4f90-9f5b-c6d61562a68e None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:31.842285 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d693de0-9346-4564-b117-8f57a692fe97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:31.843174 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d693de0-9346-4564-b117-8f57a692fe97 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:31.843453 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d693de0-9346-4564-b117-8f57a692fe97 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:31.843722 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d693de0-9346-4564-b117-8f57a692fe97 None admin] PATH_INFO: `/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:31.844967 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d693de0-9346-4564-b117-8f57a692fe97 None admin] RBAC: Authorizing `identity:get_project(project_id=8fc48df8fb1f4de59d32fefd09a8ecae)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:31.846533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d693de0-9346-4564-b117-8f57a692fe97 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:31.848974 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 131/263] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:31 2022] GET /identity/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae => generated 302 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:31.855800 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d693de0-9346-4564-b117-8f57a692fe97 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:31.977992 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe2c83-56fe-40c4-a5e9-2028d123b6a2 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:32.007299 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:32.060917 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5837184-5de6-4c2c-901a-d0a172189e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:32.061630 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5837184-5de6-4c2c-901a-d0a172189e17 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:32.061912 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5837184-5de6-4c2c-901a-d0a172189e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:32.062195 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5837184-5de6-4c2c-901a-d0a172189e17 None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:32.062967 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5837184-5de6-4c2c-901a-d0a172189e17 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:32.063275 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5837184-5de6-4c2c-901a-d0a172189e17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:32.064693 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5837184-5de6-4c2c-901a-d0a172189e17 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:32.068203 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d693de0-9346-4564-b117-8f57a692fe97 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:32.100693 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 133/264] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:02:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:32.104299 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5837184-5de6-4c2c-901a-d0a172189e17 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:32.127856 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:32.164330 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e904c3-2a0a-4af6-8e17-ac827aae1160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:32.165105 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e904c3-2a0a-4af6-8e17-ac827aae1160 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:32.165389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e904c3-2a0a-4af6-8e17-ac827aae1160 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:32.165665 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e904c3-2a0a-4af6-8e17-ac827aae1160 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:32.166585 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e904c3-2a0a-4af6-8e17-ac827aae1160 None admin] RBAC: Adding query filter params (role.id=d336dd7b746c4488b33289c00302e4d5, scope.project.id=8fc48df8fb1f4de59d32fefd09a8ecae, user.id=c73d62abc85e489b924d33d2c053ad21) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:32.166936 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e904c3-2a0a-4af6-8e17-ac827aae1160 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:32.169275 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e904c3-2a0a-4af6-8e17-ac827aae1160 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:32.188010 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 132/265] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:02:31 2022] GET /identity/v3/role_assignments?user.id=c73d62abc85e489b924d33d2c053ad21&scope.project.id=8fc48df8fb1f4de59d32fefd09a8ecae&role.id=d336dd7b746c4488b33289c00302e4d5 => generated 608 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:32.256944 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5837184-5de6-4c2c-901a-d0a172189e17 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:32.309193 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f776905-adea-454d-8df2-f5b72a2f1779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:32.310168 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f776905-adea-454d-8df2-f5b72a2f1779 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:32.310454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f776905-adea-454d-8df2-f5b72a2f1779 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:32.310730 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f776905-adea-454d-8df2-f5b72a2f1779 None admin] PATH_INFO: `/v3/users/c73d62abc85e489b924d33d2c053ad21` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:32.311995 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f776905-adea-454d-8df2-f5b72a2f1779 None admin] RBAC: Authorizing `identity:get_user(user_id=c73d62abc85e489b924d33d2c053ad21)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:32.313542 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f776905-adea-454d-8df2-f5b72a2f1779 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:32.315214 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 134/266] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:32 2022] GET /identity/v3/users/c73d62abc85e489b924d33d2c053ad21 => generated 283 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:32.320760 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e904c3-2a0a-4af6-8e17-ac827aae1160 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:32.331244 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a4e091-fa70-4537-9bf3-14b9af6c3483 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:32.331626 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a4e091-fa70-4537-9bf3-14b9af6c3483 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:32.331805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a4e091-fa70-4537-9bf3-14b9af6c3483 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:32.333315 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 135/267] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:32.342411 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebc59f-14c7-43ab-8def-143825d75c82 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:32.342703 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebc59f-14c7-43ab-8def-143825d75c82 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:32.342987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebc59f-14c7-43ab-8def-143825d75c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:32.372880 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-2cebc59f-14c7-43ab-8def-143825d75c82 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:32.374303 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-2cebc59f-14c7-43ab-8def-143825d75c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:32.444311 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:32.469584 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e904c3-2a0a-4af6-8e17-ac827aae1160 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:32.498498 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 136/268] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:32.527852 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243640d5-66ca-4847-8dff-87c95360e710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:32.528486 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243640d5-66ca-4847-8dff-87c95360e710 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:32.528770 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243640d5-66ca-4847-8dff-87c95360e710 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:32.529150 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243640d5-66ca-4847-8dff-87c95360e710 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:32.547290 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cebc59f-14c7-43ab-8def-143825d75c82 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:32.597518 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:32.614253 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243640d5-66ca-4847-8dff-87c95360e710 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:32.615970 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243640d5-66ca-4847-8dff-87c95360e710 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:32.734116 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cebc59f-14c7-43ab-8def-143825d75c82 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:32.740489 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 133/269] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:02:32.742158 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e763a7b-951d-4798-b7c8-41815158393c None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:32.742623 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e763a7b-951d-4798-b7c8-41815158393c None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:32.743053 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e763a7b-951d-4798-b7c8-41815158393c None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:32.744664 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 134/270] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:32.745600 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e763a7b-951d-4798-b7c8-41815158393c None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:32.784527 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadd2f04-5c35-4a36-a010-d01c8eea30ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:32.785718 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd2f04-5c35-4a36-a010-d01c8eea30ed None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:32.786009 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd2f04-5c35-4a36-a010-d01c8eea30ed None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:32.786287 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd2f04-5c35-4a36-a010-d01c8eea30ed None admin] PATH_INFO: `/v3/roles/member` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:32.806104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadd2f04-5c35-4a36-a010-d01c8eea30ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:32.817144 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-cadd2f04-5c35-4a36-a010-d01c8eea30ed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:32.817144 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:32.817144 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:32.817144 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:32.817144 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:32.817144 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:32.817144 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:32.817144 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:32.817144 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:32.817144 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:32.817144 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:32.817144 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:32.817144 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:32.817144 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:32.817144 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:32.817144 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:32.818307 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:32.818307 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:32.818307 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:32.818307 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:32.818307 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:32.818307 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:32.818307 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:32.818307 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:32.818307 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:32.818307 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:32.818307 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:32.818307 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:32.818307 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:32.818307 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:32.818307 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:32.818307 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:32.824069 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:32.824069 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:32.824069 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:32.824069 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:32.824069 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:32.824069 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:32.824069 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:32.824069 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:32.824069 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:32.824069 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:32.824069 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:32.824069 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:02:32.824069 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 137/271] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:02:32.824069 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d44bc6-633f-472f-aaa9-2e172d8e127f None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:32.825098 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d44bc6-633f-472f-aaa9-2e172d8e127f None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:32.825098 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d44bc6-633f-472f-aaa9-2e172d8e127f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:32.843845 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-01d44bc6-633f-472f-aaa9-2e172d8e127f None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:32.843845 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-01d44bc6-633f-472f-aaa9-2e172d8e127f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:32.859688 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:32.889647 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e763a7b-951d-4798-b7c8-41815158393c None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:32.889841 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:32.927036 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65680733-37b9-4422-b9c2-1c67c79215ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:32.932962 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65680733-37b9-4422-b9c2-1c67c79215ec None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:32.933088 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 138/272] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:32.933164 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65680733-37b9-4422-b9c2-1c67c79215ec None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:32.933164 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65680733-37b9-4422-b9c2-1c67c79215ec None admin] PATH_INFO: `/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:32.933369 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d44bc6-633f-472f-aaa9-2e172d8e127f None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:32.933467 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65680733-37b9-4422-b9c2-1c67c79215ec None admin] RBAC: Authorizing `identity:get_project(project_id=8fc48df8fb1f4de59d32fefd09a8ecae)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:32.933467 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65680733-37b9-4422-b9c2-1c67c79215ec None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:32.933467 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 135/273] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:32 2022] GET /identity/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:32.934982 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d788c09-3baf-4520-8026-f01e53ae8d5a None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:32.935266 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d788c09-3baf-4520-8026-f01e53ae8d5a None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:32.935612 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d788c09-3baf-4520-8026-f01e53ae8d5a None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:32.941057 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 136/274] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:32.942082 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d788c09-3baf-4520-8026-f01e53ae8d5a None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:33.052779 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d44bc6-633f-472f-aaa9-2e172d8e127f None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:33.065936 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d788c09-3baf-4520-8026-f01e53ae8d5a None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:33.091620 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210cfb53-907b-42ba-aaa1-57f7c6d22e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:33.092411 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210cfb53-907b-42ba-aaa1-57f7c6d22e39 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:33.092700 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210cfb53-907b-42ba-aaa1-57f7c6d22e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:33.093005 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210cfb53-907b-42ba-aaa1-57f7c6d22e39 None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:33.093754 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210cfb53-907b-42ba-aaa1-57f7c6d22e39 None admin] RBAC: Adding query filter params (name=member) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:33.094065 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210cfb53-907b-42ba-aaa1-57f7c6d22e39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:33.095608 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210cfb53-907b-42ba-aaa1-57f7c6d22e39 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:33.105312 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c809e1f-9650-4afa-917a-c08400c32ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:33.106125 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 139/275] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 27 21:02:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:33.106251 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c809e1f-9650-4afa-917a-c08400c32ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:33.107333 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3fef9-df75-4a49-a1ee-49d8cc7bf55d None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:33.107717 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3fef9-df75-4a49-a1ee-49d8cc7bf55d None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:33.107998 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3fef9-df75-4a49-a1ee-49d8cc7bf55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:33.111221 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c809e1f-9650-4afa-917a-c08400c32ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:33.111515 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c809e1f-9650-4afa-917a-c08400c32ab4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:33.112413 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c809e1f-9650-4afa-917a-c08400c32ab4 None admin] RBAC: Adding query filter params (role.id=5834784bf54d4806b88e32e0fb8e18fe, scope.project.id=8fc48df8fb1f4de59d32fefd09a8ecae, user.id=c73d62abc85e489b924d33d2c053ad21) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:33.114339 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c809e1f-9650-4afa-917a-c08400c32ab4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:33.114339 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c809e1f-9650-4afa-917a-c08400c32ab4 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:33.122220 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 137/276] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:02:32 2022] GET /identity/v3/role_assignments?user.id=c73d62abc85e489b924d33d2c053ad21&scope.project.id=8fc48df8fb1f4de59d32fefd09a8ecae&role.id=5834784bf54d4806b88e32e0fb8e18fe => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:33.123222 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c809e1f-9650-4afa-917a-c08400c32ab4 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:33.127898 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-e9c3fef9-df75-4a49-a1ee-49d8cc7bf55d None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:33.129225 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-e9c3fef9-df75-4a49-a1ee-49d8cc7bf55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:33.150942 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:33.166459 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:33.220780 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 140/277] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:33.222202 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c3fef9-df75-4a49-a1ee-49d8cc7bf55d None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:33.301686 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c809e1f-9650-4afa-917a-c08400c32ab4 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:33.349650 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf67dc1-f335-45df-a84c-48a32dc4c09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:33.350829 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf67dc1-f335-45df-a84c-48a32dc4c09f None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:33.351308 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf67dc1-f335-45df-a84c-48a32dc4c09f None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:33.351794 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf67dc1-f335-45df-a84c-48a32dc4c09f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:33.353820 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf67dc1-f335-45df-a84c-48a32dc4c09f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:33.355415 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf67dc1-f335-45df-a84c-48a32dc4c09f None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:33.357490 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 138/278] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:02:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:33.358285 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf67dc1-f335-45df-a84c-48a32dc4c09f None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:33.378278 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c3fef9-df75-4a49-a1ee-49d8cc7bf55d None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:33.425388 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b881719-bcea-4b3f-8bcd-3b38dd6044d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:33.426289 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b881719-bcea-4b3f-8bcd-3b38dd6044d0 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:33.426562 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b881719-bcea-4b3f-8bcd-3b38dd6044d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:33.426835 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b881719-bcea-4b3f-8bcd-3b38dd6044d0 None admin] PATH_INFO: `/v3/users/09041e7213a84b91a459bef462360d2f` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:33.446432 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b881719-bcea-4b3f-8bcd-3b38dd6044d0 None admin] RBAC: Authorizing `identity:get_user(user_id=09041e7213a84b91a459bef462360d2f)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:33.448510 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b881719-bcea-4b3f-8bcd-3b38dd6044d0 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:33.456200 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 141/279] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:33 2022] GET /identity/v3/users/09041e7213a84b91a459bef462360d2f => generated 293 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:33.456200 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b881719-bcea-4b3f-8bcd-3b38dd6044d0 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:33.457441 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:33.478904 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf67dc1-f335-45df-a84c-48a32dc4c09f None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:33.482440 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:33.523098 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff0b49-8394-4243-aa88-d88a427f8b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:33.524239 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff0b49-8394-4243-aa88-d88a427f8b5b None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:33.524572 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff0b49-8394-4243-aa88-d88a427f8b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:33.524861 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff0b49-8394-4243-aa88-d88a427f8b5b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:33.541632 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aff0b49-8394-4243-aa88-d88a427f8b5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:33.551440 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-5aff0b49-8394-4243-aa88-d88a427f8b5b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 21:02:33.551440 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:33.551440 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:33.551440 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:33.551440 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:33.551440 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:33.551440 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:33.551440 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:33.551440 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:33.551440 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:33.551440 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:33.551440 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:33.551440 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:33.551440 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:33.551440 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:33.551440 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:33.552401 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:33.552401 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:33.552401 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:33.552401 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:33.552401 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:33.552401 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:33.552401 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:33.552401 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:33.552401 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:33.552401 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:33.552401 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:33.552401 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:33.552401 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:33.552401 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:33.552401 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:33.552401 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:33.553235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:33.553235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:33.553235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:33.553235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:33.553235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:33.553235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:33.553235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:33.553235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:33.553235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:33.553235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:33.553235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 21:02:33.553235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:02:33.553235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 139/280] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:33.553910 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff0b49-8394-4243-aa88-d88a427f8b5b None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:33.598508 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b881719-bcea-4b3f-8bcd-3b38dd6044d0 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:33.642679 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aada7be-d94c-4781-ac31-dc8b696b47e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:33.643429 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aada7be-d94c-4781-ac31-dc8b696b47e4 None admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:33.643708 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aada7be-d94c-4781-ac31-dc8b696b47e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:33.643994 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aada7be-d94c-4781-ac31-dc8b696b47e4 None admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:33.644693 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aada7be-d94c-4781-ac31-dc8b696b47e4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:33.646180 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aada7be-d94c-4781-ac31-dc8b696b47e4 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:33.670139 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 142/281] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Jun 27 21:02:33 2022] POST /identity/v3/users => generated 293 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:02:33.670930 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aada7be-d94c-4781-ac31-dc8b696b47e4 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:33.695181 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff0b49-8394-4243-aa88-d88a427f8b5b None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:33.742071 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8568a82-4d25-4e46-8631-8488888a100a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:33.742769 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8568a82-4d25-4e46-8631-8488888a100a None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:33.743054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8568a82-4d25-4e46-8631-8488888a100a None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:33.743328 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8568a82-4d25-4e46-8631-8488888a100a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:33.744348 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8568a82-4d25-4e46-8631-8488888a100a None admin] RBAC: Adding query filter params (role.id=e4414bde4df44de8aacb36c585a93e07, user.id=09041e7213a84b91a459bef462360d2f, scope.system=all) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:33.744847 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8568a82-4d25-4e46-8631-8488888a100a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:33.746927 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8568a82-4d25-4e46-8631-8488888a100a None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:33.753493 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 140/282] 10.222.0.13 () {56 vars in 1308 bytes} [Mon Jun 27 21:02:33 2022] GET /identity/v3/role_assignments?user.id=09041e7213a84b91a459bef462360d2f&scope.system=all&role.id=e4414bde4df44de8aacb36c585a93e07 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:33.763114 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7e3bc-ba59-4078-9f63-f2670b271cef None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:33.763806 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7e3bc-ba59-4078-9f63-f2670b271cef None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:33.764257 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7e3bc-ba59-4078-9f63-f2670b271cef None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:33.766536 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 141/283] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:33.767647 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:33.772905 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af85dd-8db4-4e1a-86e0-f4eb6cec87b5 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:33.773258 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af85dd-8db4-4e1a-86e0-f4eb6cec87b5 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:33.773523 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af85dd-8db4-4e1a-86e0-f4eb6cec87b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:33.786708 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:33.794360 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-44af85dd-8db4-4e1a-86e0-f4eb6cec87b5 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:33.795758 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-44af85dd-8db4-4e1a-86e0-f4eb6cec87b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:33.798127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aada7be-d94c-4781-ac31-dc8b696b47e4 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:33.837690 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ae134-739e-4133-b5cb-24431b920a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:33.838393 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ae134-739e-4133-b5cb-24431b920a91 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:33.838676 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ae134-739e-4133-b5cb-24431b920a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:33.838945 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ae134-739e-4133-b5cb-24431b920a91 None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:33.839907 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432ae134-739e-4133-b5cb-24431b920a91 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:33.840179 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432ae134-739e-4133-b5cb-24431b920a91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:33.841702 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432ae134-739e-4133-b5cb-24431b920a91 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:33.851682 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 143/284] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 27 21:02:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:33.861625 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ae134-739e-4133-b5cb-24431b920a91 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:33.885157 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 142/285] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:02:33.911457 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af85dd-8db4-4e1a-86e0-f4eb6cec87b5 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:33.992173 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ae134-739e-4133-b5cb-24431b920a91 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:34.042238 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6545c-6c99-4304-9224-d6135a3a5842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:34.042994 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af85dd-8db4-4e1a-86e0-f4eb6cec87b5 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:34.044155 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6545c-6c99-4304-9224-d6135a3a5842 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:34.044651 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6545c-6c99-4304-9224-d6135a3a5842 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:34.045352 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6545c-6c99-4304-9224-d6135a3a5842 None admin] PATH_INFO: `/v3/users/0ed54725ccd84ea19e42fb7a87144676` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:34.049010 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be6545c-6c99-4304-9224-d6135a3a5842 None admin] RBAC: Authorizing `identity:get_user(user_id=0ed54725ccd84ea19e42fb7a87144676)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:34.050816 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be6545c-6c99-4304-9224-d6135a3a5842 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:34.052697 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 144/286] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:33 2022] GET /identity/v3/users/0ed54725ccd84ea19e42fb7a87144676 => generated 297 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:34.058570 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6545c-6c99-4304-9224-d6135a3a5842 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:34.075172 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:34.097779 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d214b-792a-4833-9a9f-034fe74b16ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:34.098712 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d214b-792a-4833-9a9f-034fe74b16ff None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:34.098994 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d214b-792a-4833-9a9f-034fe74b16ff None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:34.099367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d214b-792a-4833-9a9f-034fe74b16ff None admin] PATH_INFO: `/v3/roles/member` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:34.105268 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:34.113134 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4d214b-792a-4833-9a9f-034fe74b16ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:34.121419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-aa4d214b-792a-4833-9a9f-034fe74b16ff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:34.121419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:34.121419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:34.121419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:34.121419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:34.121419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:34.121419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:34.121419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:34.121419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:34.121419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:34.121419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:34.121419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:34.121419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:34.121419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:34.121419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:34.121419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:34.122394 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:34.122394 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:34.122394 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:34.122394 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:34.122394 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:34.122394 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:34.122394 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:34.122394 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:34.122394 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:34.122394 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:34.122394 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:34.122394 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:34.122394 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:34.122394 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:34.122394 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:34.122394 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:34.123176 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:34.123176 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:34.123176 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:34.123176 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:34.123176 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:34.123176 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:34.123176 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:34.123176 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:34.123176 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:34.123176 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:34.123176 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:34.123176 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:02:34.123176 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 143/287] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:02:34.129126 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d214b-792a-4833-9a9f-034fe74b16ff None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:34.213466 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6545c-6c99-4304-9224-d6135a3a5842 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:34.242460 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d214b-792a-4833-9a9f-034fe74b16ff None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:34.264773 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9409a9-dfa9-4197-840b-b6ea2da718af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:34.267053 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9409a9-dfa9-4197-840b-b6ea2da718af None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:34.268323 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9409a9-dfa9-4197-840b-b6ea2da718af None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:34.268950 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9409a9-dfa9-4197-840b-b6ea2da718af None admin] PATH_INFO: `/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:34.272489 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9409a9-dfa9-4197-840b-b6ea2da718af None admin] RBAC: Authorizing `identity:get_project(project_id=8fc48df8fb1f4de59d32fefd09a8ecae)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:34.277363 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9409a9-dfa9-4197-840b-b6ea2da718af None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:34.282446 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 145/288] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:34 2022] GET /identity/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:34.287454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9409a9-dfa9-4197-840b-b6ea2da718af None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:34.304864 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab11c66-9c7a-45e1-a96e-7727a77c849a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:34.305860 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab11c66-9c7a-45e1-a96e-7727a77c849a None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:34.306142 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab11c66-9c7a-45e1-a96e-7727a77c849a None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:34.306412 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab11c66-9c7a-45e1-a96e-7727a77c849a None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:34.307172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab11c66-9c7a-45e1-a96e-7727a77c849a None admin] RBAC: Adding query filter params (name=member) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:34.307477 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab11c66-9c7a-45e1-a96e-7727a77c849a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:34.310462 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab11c66-9c7a-45e1-a96e-7727a77c849a None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:34.328299 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 144/289] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 27 21:02:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:34.335652 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab11c66-9c7a-45e1-a96e-7727a77c849a None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:34.409573 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:34.437201 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:34.461400 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9409a9-dfa9-4197-840b-b6ea2da718af None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:34.485820 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab11c66-9c7a-45e1-a96e-7727a77c849a None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:34.522306 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a3ca49-57cd-43fd-b0fb-a91b626195f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:34.523537 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3ca49-57cd-43fd-b0fb-a91b626195f0 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:34.523993 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3ca49-57cd-43fd-b0fb-a91b626195f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:34.524434 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3ca49-57cd-43fd-b0fb-a91b626195f0 None admin] PATH_INFO: `/v3/users/5ebbceb49ff04c3f841e47c3aaa9d960` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:34.525849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a3ca49-57cd-43fd-b0fb-a91b626195f0 None admin] RBAC: Authorizing `identity:get_user(user_id=5ebbceb49ff04c3f841e47c3aaa9d960)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:34.527943 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a3ca49-57cd-43fd-b0fb-a91b626195f0 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:34.528466 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb0cc7-5d23-4c45-8bc3-2b759ee536b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:34.529458 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb0cc7-5d23-4c45-8bc3-2b759ee536b5 None admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:34.529761 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb0cc7-5d23-4c45-8bc3-2b759ee536b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:34.530031 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb0cc7-5d23-4c45-8bc3-2b759ee536b5 None admin] PATH_INFO: `/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae/users/0ed54725ccd84ea19e42fb7a87144676/roles/df30c7d320e947eaabb270652cb4afa3` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:34.530668 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 145/290] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:34 2022] GET /identity/v3/users/5ebbceb49ff04c3f841e47c3aaa9d960 => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:34.540949 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a3ca49-57cd-43fd-b0fb-a91b626195f0 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:34.541090 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becb0cc7-5d23-4c45-8bc3-2b759ee536b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=8fc48df8fb1f4de59d32fefd09a8ecae, user_id=0ed54725ccd84ea19e42fb7a87144676, role_id=df30c7d320e947eaabb270652cb4afa3)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:34.541090 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becb0cc7-5d23-4c45-8bc3-2b759ee536b5 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:34.564776 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 146/291] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 27 21:02:34 2022] PUT /identity/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae/users/0ed54725ccd84ea19e42fb7a87144676/roles/df30c7d320e947eaabb270652cb4afa3 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:02:34.710830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a3ca49-57cd-43fd-b0fb-a91b626195f0 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:34.752047 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:34.762969 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31348462-57e7-4a63-9f8c-d20a01b219b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:34.763676 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31348462-57e7-4a63-9f8c-d20a01b219b5 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:34.763956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31348462-57e7-4a63-9f8c-d20a01b219b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:34.764227 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31348462-57e7-4a63-9f8c-d20a01b219b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:34.869174 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31348462-57e7-4a63-9f8c-d20a01b219b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:34.870838 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31348462-57e7-4a63-9f8c-d20a01b219b5 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:35.000897 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 146/292] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:02:35.009037 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb0cc7-5d23-4c45-8bc3-2b759ee536b5 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:35.127506 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb0cc7-5d23-4c45-8bc3-2b759ee536b5 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:35.169054 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2422cfd6-aba0-4706-8a8a-95ecde1110b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:35.170044 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2422cfd6-aba0-4706-8a8a-95ecde1110b3 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:35.170327 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2422cfd6-aba0-4706-8a8a-95ecde1110b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:35.170623 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2422cfd6-aba0-4706-8a8a-95ecde1110b3 None admin] PATH_INFO: `/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:35.172032 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2422cfd6-aba0-4706-8a8a-95ecde1110b3 None admin] RBAC: Authorizing `identity:get_project(project_id=8fc48df8fb1f4de59d32fefd09a8ecae)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:35.173664 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2422cfd6-aba0-4706-8a8a-95ecde1110b3 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:35.176064 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 147/293] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:35 2022] GET /identity/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:35.182480 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31348462-57e7-4a63-9f8c-d20a01b219b5 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:35.222911 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:35.305148 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31348462-57e7-4a63-9f8c-d20a01b219b5 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:35.345874 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe8adb4-690c-472c-b549-8aa4945c9b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:35.346546 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe8adb4-690c-472c-b549-8aa4945c9b23 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:35.346879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe8adb4-690c-472c-b549-8aa4945c9b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:35.347155 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe8adb4-690c-472c-b549-8aa4945c9b23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:35.348005 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe8adb4-690c-472c-b549-8aa4945c9b23 None admin] RBAC: Adding query filter params (role.id=e4414bde4df44de8aacb36c585a93e07, scope.project.id=8fc48df8fb1f4de59d32fefd09a8ecae, user.id=5ebbceb49ff04c3f841e47c3aaa9d960) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:35.348318 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe8adb4-690c-472c-b549-8aa4945c9b23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:35.349803 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe8adb4-690c-472c-b549-8aa4945c9b23 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:35.356857 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 147/294] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:02:35 2022] GET /identity/v3/role_assignments?user.id=5ebbceb49ff04c3f841e47c3aaa9d960&scope.project.id=8fc48df8fb1f4de59d32fefd09a8ecae&role.id=e4414bde4df44de8aacb36c585a93e07 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:36.191563 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1158355-3320-411d-86a4-e31b6eeddd60 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:36.192625 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1158355-3320-411d-86a4-e31b6eeddd60 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:36.193152 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1158355-3320-411d-86a4-e31b6eeddd60 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:36.194990 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 148/295] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:36.200590 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b737ad8-32f0-4d67-ae3f-d52f02d4517a None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:36.201471 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b737ad8-32f0-4d67-ae3f-d52f02d4517a None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:36.201987 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b737ad8-32f0-4d67-ae3f-d52f02d4517a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:36.208953 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3479b7a-02ab-49e5-a82e-d9e1f795ee75 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:36.209234 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3479b7a-02ab-49e5-a82e-d9e1f795ee75 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:36.209508 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3479b7a-02ab-49e5-a82e-d9e1f795ee75 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:36.211131 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 149/296] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:02:36.216424 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7aa7ab-cc11-45be-828b-74b21051ceb1 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:36.216698 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7aa7ab-cc11-45be-828b-74b21051ceb1 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:36.216995 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7aa7ab-cc11-45be-828b-74b21051ceb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:36.221753 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-8b737ad8-32f0-4d67-ae3f-d52f02d4517a None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:36.222209 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:36.223116 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-8b737ad8-32f0-4d67-ae3f-d52f02d4517a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:36.242283 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-5e7aa7ab-cc11-45be-828b-74b21051ceb1 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:36.244159 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-5e7aa7ab-cc11-45be-828b-74b21051ceb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:36.309088 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 148/297] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:36.326842 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 150/298] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:02:36.336407 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b737ad8-32f0-4d67-ae3f-d52f02d4517a None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:36.349478 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7aa7ab-cc11-45be-828b-74b21051ceb1 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:36.360719 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:36.447383 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b737ad8-32f0-4d67-ae3f-d52f02d4517a None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:36.463358 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7aa7ab-cc11-45be-828b-74b21051ceb1 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:36.483573 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d38e3-85df-4c4a-b9a1-6848b5cb0ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:36.484382 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d38e3-85df-4c4a-b9a1-6848b5cb0ab3 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:36.484662 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d38e3-85df-4c4a-b9a1-6848b5cb0ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:36.484955 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d38e3-85df-4c4a-b9a1-6848b5cb0ab3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:36.493575 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:36.496797 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14d38e3-85df-4c4a-b9a1-6848b5cb0ab3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:36.498993 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab8420-3f0f-4f29-a050-529b6245fed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:36.499797 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab8420-3f0f-4f29-a050-529b6245fed7 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:36.500077 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab8420-3f0f-4f29-a050-529b6245fed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:36.500345 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab8420-3f0f-4f29-a050-529b6245fed7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:36.504562 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-f14d38e3-85df-4c4a-b9a1-6848b5cb0ab3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 21:02:36.504562 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:36.504562 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:36.504562 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:36.504562 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:36.504562 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:36.504562 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:36.504562 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:36.504562 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:36.504562 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:36.504562 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:36.504562 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:36.504562 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:36.504562 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:36.504562 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:36.504562 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:36.505464 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:36.505464 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:36.505464 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:36.505464 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:36.505464 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:36.505464 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:36.505464 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:36.505464 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:36.505464 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:36.505464 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:36.505464 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:36.505464 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:36.505464 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:36.505464 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:36.505464 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:36.505464 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:36.506263 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:36.506263 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:36.506263 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:36.506263 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:36.506263 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:36.506263 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:36.506263 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:36.506263 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:36.506263 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:36.506263 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:36.506263 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 21:02:36.506263 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:02:36.506263 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 149/299] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:36.507447 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf48d63-223b-4f18-8ff1-d4bd0d2fe5f1 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:36.507739 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf48d63-223b-4f18-8ff1-d4bd0d2fe5f1 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:36.508014 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf48d63-223b-4f18-8ff1-d4bd0d2fe5f1 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:36.512966 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 150/300] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:36.512966 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf48d63-223b-4f18-8ff1-d4bd0d2fe5f1 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:36.519920 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ab8420-3f0f-4f29-a050-529b6245fed7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:36.527737 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-56ab8420-3f0f-4f29-a050-529b6245fed7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:36.527737 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:36.527737 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:36.527737 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:36.527737 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:36.527737 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:36.527737 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:36.527737 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:36.527737 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:36.527737 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:36.527737 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:36.527737 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:36.527737 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:36.527737 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:36.527737 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:36.527737 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:36.528641 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:36.528641 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:36.528641 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:36.528641 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:36.528641 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:36.528641 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:36.528641 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:36.528641 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:36.528641 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:36.528641 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:36.528641 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:36.528641 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:36.528641 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:36.528641 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:36.528641 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:36.528641 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:36.529472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:36.529472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:36.529472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:36.529472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:36.529472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:36.529472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:36.529472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:36.529472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:36.529472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:36.529472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:36.529472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:36.529472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:02:36.529472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 151/301] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:02:36.534267 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f4884-64da-4ceb-afcf-aefd32261775 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:36.534709 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f4884-64da-4ceb-afcf-aefd32261775 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:36.534990 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f4884-64da-4ceb-afcf-aefd32261775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:36.552202 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-600f4884-64da-4ceb-afcf-aefd32261775 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:36.553461 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-600f4884-64da-4ceb-afcf-aefd32261775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:36.605449 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:36.624984 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 152/302] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:36.625318 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600f4884-64da-4ceb-afcf-aefd32261775 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:36.632722 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf48d63-223b-4f18-8ff1-d4bd0d2fe5f1 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:36.671227 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f69e5f-7ab3-4976-a6e9-53bd8ddbba1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:36.671908 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f69e5f-7ab3-4976-a6e9-53bd8ddbba1f None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:36.672188 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f69e5f-7ab3-4976-a6e9-53bd8ddbba1f None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:36.672458 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f69e5f-7ab3-4976-a6e9-53bd8ddbba1f None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:36.673251 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f69e5f-7ab3-4976-a6e9-53bd8ddbba1f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:36.673557 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f69e5f-7ab3-4976-a6e9-53bd8ddbba1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:36.675140 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f69e5f-7ab3-4976-a6e9-53bd8ddbba1f None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:36.689927 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 151/303] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 27 21:02:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:36.689927 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f69e5f-7ab3-4976-a6e9-53bd8ddbba1f None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:36.744026 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600f4884-64da-4ceb-afcf-aefd32261775 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:36.768207 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:36.781550 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8483b97-4294-463f-afce-271d063fb6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:36.782301 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8483b97-4294-463f-afce-271d063fb6c8 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:36.782593 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8483b97-4294-463f-afce-271d063fb6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:36.782872 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8483b97-4294-463f-afce-271d063fb6c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:36.783631 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8483b97-4294-463f-afce-271d063fb6c8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:36.783939 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8483b97-4294-463f-afce-271d063fb6c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:36.785371 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8483b97-4294-463f-afce-271d063fb6c8 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:36.793802 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 153/304] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 27 21:02:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:36.793802 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8483b97-4294-463f-afce-271d063fb6c8 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:36.820064 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f69e5f-7ab3-4976-a6e9-53bd8ddbba1f None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:36.867472 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5acee4a-e4eb-493e-8788-166d560de7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:36.868607 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5acee4a-e4eb-493e-8788-166d560de7cc None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:36.868939 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5acee4a-e4eb-493e-8788-166d560de7cc None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:36.869404 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5acee4a-e4eb-493e-8788-166d560de7cc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:36.872421 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5acee4a-e4eb-493e-8788-166d560de7cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:36.874257 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5acee4a-e4eb-493e-8788-166d560de7cc None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:36.881580 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 152/305] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:02:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:36.881580 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5acee4a-e4eb-493e-8788-166d560de7cc None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:36.899522 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:36.902287 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8483b97-4294-463f-afce-271d063fb6c8 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:36.940835 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dfb4ac-d14b-48a4-9ddb-397d24ffe5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:36.941888 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dfb4ac-d14b-48a4-9ddb-397d24ffe5d5 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:36.942289 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dfb4ac-d14b-48a4-9ddb-397d24ffe5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:36.942552 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dfb4ac-d14b-48a4-9ddb-397d24ffe5d5 None admin] PATH_INFO: `/v3/users/8bed4e30d52b477abdc9fe06ff367c61` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:36.957354 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dfb4ac-d14b-48a4-9ddb-397d24ffe5d5 None admin] RBAC: Authorizing `identity:get_user(user_id=8bed4e30d52b477abdc9fe06ff367c61)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:36.958827 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dfb4ac-d14b-48a4-9ddb-397d24ffe5d5 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:36.960819 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 154/306] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:36 2022] GET /identity/v3/users/8bed4e30d52b477abdc9fe06ff367c61 => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:36.961479 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dfb4ac-d14b-48a4-9ddb-397d24ffe5d5 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:36.999889 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5acee4a-e4eb-493e-8788-166d560de7cc None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:37.008425 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:37.041005 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd69a0-3f1b-4f5e-be28-717eb7e16c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:37.041895 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd69a0-3f1b-4f5e-be28-717eb7e16c46 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:37.042172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd69a0-3f1b-4f5e-be28-717eb7e16c46 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:37.042441 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd69a0-3f1b-4f5e-be28-717eb7e16c46 None admin] PATH_INFO: `/v3/users/09041e7213a84b91a459bef462360d2f` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:37.043897 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cd69a0-3f1b-4f5e-be28-717eb7e16c46 None admin] RBAC: Authorizing `identity:get_user(user_id=09041e7213a84b91a459bef462360d2f)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:37.045390 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cd69a0-3f1b-4f5e-be28-717eb7e16c46 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:37.047057 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 153/307] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:36 2022] GET /identity/v3/users/09041e7213a84b91a459bef462360d2f => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:37.052541 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd69a0-3f1b-4f5e-be28-717eb7e16c46 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:37.072837 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dfb4ac-d14b-48a4-9ddb-397d24ffe5d5 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:37.112538 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337ccd9c-9258-4273-b919-ba317d3f58de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:37.113519 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ccd9c-9258-4273-b919-ba317d3f58de None admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:37.113519 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ccd9c-9258-4273-b919-ba317d3f58de None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:37.113802 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ccd9c-9258-4273-b919-ba317d3f58de None admin] PATH_INFO: `/v3/groups` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:37.114592 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337ccd9c-9258-4273-b919-ba317d3f58de None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:37.115943 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337ccd9c-9258-4273-b919-ba317d3f58de None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:37.141046 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 155/308] 10.222.0.13 () {60 vars in 1134 bytes} [Mon Jun 27 21:02:36 2022] POST /identity/v3/groups => generated 229 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:02:37.141046 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337ccd9c-9258-4273-b919-ba317d3f58de None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:37.146348 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:37.176553 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd69a0-3f1b-4f5e-be28-717eb7e16c46 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:37.225194 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33968b7-54be-4c48-8f9e-8bdc1b0e179f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:37.226017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33968b7-54be-4c48-8f9e-8bdc1b0e179f None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:37.226190 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33968b7-54be-4c48-8f9e-8bdc1b0e179f None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:37.226491 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33968b7-54be-4c48-8f9e-8bdc1b0e179f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:37.227719 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33968b7-54be-4c48-8f9e-8bdc1b0e179f None admin] RBAC: Adding query filter params (role.id=df30c7d320e947eaabb270652cb4afa3, user.id=8bed4e30d52b477abdc9fe06ff367c61, scope.system=all) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:37.228036 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33968b7-54be-4c48-8f9e-8bdc1b0e179f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:37.229985 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33968b7-54be-4c48-8f9e-8bdc1b0e179f None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:37.236543 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 154/309] 10.222.0.13 () {56 vars in 1308 bytes} [Mon Jun 27 21:02:37 2022] GET /identity/v3/role_assignments?user.id=8bed4e30d52b477abdc9fe06ff367c61&scope.system=all&role.id=df30c7d320e947eaabb270652cb4afa3 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:37.237819 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ecd15-e0d8-40fa-bacb-58c752101843 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:37.238085 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ecd15-e0d8-40fa-bacb-58c752101843 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:37.238396 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ecd15-e0d8-40fa-bacb-58c752101843 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:37.240078 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 155/310] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:37.246627 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8149cc4-6ede-4098-bc44-a44993bfb2e7 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:37.247014 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8149cc4-6ede-4098-bc44-a44993bfb2e7 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:37.247279 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8149cc4-6ede-4098-bc44-a44993bfb2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:37.266522 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-d8149cc4-6ede-4098-bc44-a44993bfb2e7 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:37.267822 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d8149cc4-6ede-4098-bc44-a44993bfb2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:37.277019 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337ccd9c-9258-4273-b919-ba317d3f58de None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:37.279842 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:37.322438 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64418d1-2c2b-470f-980c-7ecb6282c9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:37.323909 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64418d1-2c2b-470f-980c-7ecb6282c9e8 None admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:37.324467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64418d1-2c2b-470f-980c-7ecb6282c9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:37.325020 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64418d1-2c2b-470f-980c-7ecb6282c9e8 None admin] PATH_INFO: `/v3/system/users/09041e7213a84b91a459bef462360d2f/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:37.327351 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64418d1-2c2b-470f-980c-7ecb6282c9e8 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=09041e7213a84b91a459bef462360d2f, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:37.329110 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64418d1-2c2b-470f-980c-7ecb6282c9e8 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:37.340298 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 156/311] 10.222.0.13 () {58 vars in 1334 bytes} [Mon Jun 27 21:02:37 2022] PUT /identity/v3/system/users/09041e7213a84b91a459bef462360d2f/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:02:37.354796 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 156/312] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:02:37.379131 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64418d1-2c2b-470f-980c-7ecb6282c9e8 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:37.518081 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64418d1-2c2b-470f-980c-7ecb6282c9e8 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:37.560873 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826df891-b122-40fd-9889-fd6da6a3a97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:37.561970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826df891-b122-40fd-9889-fd6da6a3a97a None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:37.562396 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826df891-b122-40fd-9889-fd6da6a3a97a None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:37.562836 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826df891-b122-40fd-9889-fd6da6a3a97a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:37.576144 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826df891-b122-40fd-9889-fd6da6a3a97a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:37.584366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-826df891-b122-40fd-9889-fd6da6a3a97a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 21:02:37.584366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:37.584366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:37.584366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:37.584366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:37.584366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:37.584366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:37.584366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:37.584366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:37.584366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:37.584366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:37.584366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:37.584366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:37.584366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:37.584366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:37.584366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:37.585564 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:37.585564 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:37.585564 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:37.585564 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:37.585564 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:37.585564 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:37.585564 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:37.585564 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:37.585564 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:37.585564 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:37.585564 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:37.585564 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:37.585564 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:37.585564 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:37.585564 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:37.585564 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:37.586551 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:37.586551 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:37.586551 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:37.586551 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:37.586551 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:37.586551 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:37.586551 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:37.586551 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:37.586551 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:37.586551 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:37.586551 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 21:02:37.586551 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:02:37.589011 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 157/313] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:37.596946 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8149cc4-6ede-4098-bc44-a44993bfb2e7 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:37.669267 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:37.709654 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8149cc4-6ede-4098-bc44-a44993bfb2e7 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:37.756176 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e8bee6-f638-49f3-8c8b-31d2248618a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:37.756990 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8bee6-f638-49f3-8c8b-31d2248618a5 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:37.757381 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8bee6-f638-49f3-8c8b-31d2248618a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:37.757750 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e8bee6-f638-49f3-8c8b-31d2248618a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:37.758830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e8bee6-f638-49f3-8c8b-31d2248618a5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:37.759249 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e8bee6-f638-49f3-8c8b-31d2248618a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:37.761067 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e8bee6-f638-49f3-8c8b-31d2248618a5 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:37.772382 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 157/314] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 27 21:02:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:37.777464 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826df891-b122-40fd-9889-fd6da6a3a97a None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:37.806965 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:37.888051 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826df891-b122-40fd-9889-fd6da6a3a97a None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:37.922423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c31b5b2-a9ff-4fd1-9b92-9fd9b38f9c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:37.923307 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31b5b2-a9ff-4fd1-9b92-9fd9b38f9c4f None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:37.923583 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31b5b2-a9ff-4fd1-9b92-9fd9b38f9c4f None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:37.923852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31b5b2-a9ff-4fd1-9b92-9fd9b38f9c4f None admin] PATH_INFO: `/v3/users/0ed54725ccd84ea19e42fb7a87144676` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:37.925081 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c31b5b2-a9ff-4fd1-9b92-9fd9b38f9c4f None admin] RBAC: Authorizing `identity:get_user(user_id=0ed54725ccd84ea19e42fb7a87144676)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:37.926546 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c31b5b2-a9ff-4fd1-9b92-9fd9b38f9c4f None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:37.928497 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 158/315] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:37 2022] GET /identity/v3/users/0ed54725ccd84ea19e42fb7a87144676 => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:37.934060 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e8bee6-f638-49f3-8c8b-31d2248618a5 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:38.047568 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e8bee6-f638-49f3-8c8b-31d2248618a5 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:38.087887 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6003f7-264f-4748-a03e-a41cff248559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:38.088553 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6003f7-264f-4748-a03e-a41cff248559 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:38.088833 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6003f7-264f-4748-a03e-a41cff248559 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:38.089133 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6003f7-264f-4748-a03e-a41cff248559 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:38.092072 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:38.165261 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6003f7-264f-4748-a03e-a41cff248559 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:38.167069 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6003f7-264f-4748-a03e-a41cff248559 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:38.284708 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 158/316] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:02:38.290493 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c31b5b2-a9ff-4fd1-9b92-9fd9b38f9c4f None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:38.397575 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:38.400961 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c31b5b2-a9ff-4fd1-9b92-9fd9b38f9c4f None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:38.442417 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dd328c-12bd-4af8-a4d1-cde394d35c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:38.443418 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd328c-12bd-4af8-a4d1-cde394d35c66 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:38.443726 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd328c-12bd-4af8-a4d1-cde394d35c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:38.444041 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd328c-12bd-4af8-a4d1-cde394d35c66 None admin] PATH_INFO: `/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:38.445374 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dd328c-12bd-4af8-a4d1-cde394d35c66 None admin] RBAC: Authorizing `identity:get_project(project_id=8fc48df8fb1f4de59d32fefd09a8ecae)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:38.446951 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dd328c-12bd-4af8-a4d1-cde394d35c66 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:38.449541 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 159/317] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:38 2022] GET /identity/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:38.454997 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6003f7-264f-4748-a03e-a41cff248559 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:38.495256 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:38.566538 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6003f7-264f-4748-a03e-a41cff248559 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:38.603117 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f142b89-b8cb-4a59-ad64-938ec2865726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:38.603794 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f142b89-b8cb-4a59-ad64-938ec2865726 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:38.604068 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f142b89-b8cb-4a59-ad64-938ec2865726 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:38.604337 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f142b89-b8cb-4a59-ad64-938ec2865726 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:38.605384 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f142b89-b8cb-4a59-ad64-938ec2865726 None admin] RBAC: Adding query filter params (role.id=df30c7d320e947eaabb270652cb4afa3, scope.project.id=8fc48df8fb1f4de59d32fefd09a8ecae, user.id=0ed54725ccd84ea19e42fb7a87144676) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:38.606344 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f142b89-b8cb-4a59-ad64-938ec2865726 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:38.607850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f142b89-b8cb-4a59-ad64-938ec2865726 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:38.615333 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 159/318] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:02:38 2022] GET /identity/v3/role_assignments?user.id=0ed54725ccd84ea19e42fb7a87144676&scope.project.id=8fc48df8fb1f4de59d32fefd09a8ecae&role.id=df30c7d320e947eaabb270652cb4afa3 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:39.345818 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e724cc9c-99d0-4b02-a133-fd67d99e7b1a None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:39.346329 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e724cc9c-99d0-4b02-a133-fd67d99e7b1a None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:39.346329 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e724cc9c-99d0-4b02-a133-fd67d99e7b1a None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:39.347820 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 160/319] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:39.353567 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9321fb-d12e-4744-aede-ed422edbff58 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:39.354007 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9321fb-d12e-4744-aede-ed422edbff58 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:39.354079 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9321fb-d12e-4744-aede-ed422edbff58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:39.373193 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-ee9321fb-d12e-4744-aede-ed422edbff58 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:39.374859 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ee9321fb-d12e-4744-aede-ed422edbff58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:39.419877 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d68e67-0e4c-4774-96e4-4f5a6da169d4 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:39.420273 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d68e67-0e4c-4774-96e4-4f5a6da169d4 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:39.420698 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d68e67-0e4c-4774-96e4-4f5a6da169d4 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:39.422659 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 161/320] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:39.428354 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022a3bd-477f-454f-93ed-0f7e75a6f17c None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:39.428640 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022a3bd-477f-454f-93ed-0f7e75a6f17c None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:39.428913 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022a3bd-477f-454f-93ed-0f7e75a6f17c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:39.449123 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-6022a3bd-477f-454f-93ed-0f7e75a6f17c None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:39.451103 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-6022a3bd-477f-454f-93ed-0f7e75a6f17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:39.470506 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 160/321] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:39.493187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9321fb-d12e-4744-aede-ed422edbff58 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:39.523346 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:39.529232 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 162/322] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:39.551558 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6022a3bd-477f-454f-93ed-0f7e75a6f17c None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:39.603066 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9321fb-d12e-4744-aede-ed422edbff58 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:39.636399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f82dfe-3f91-4909-a717-9e0be1915f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:39.637371 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f82dfe-3f91-4909-a717-9e0be1915f97 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:39.637647 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f82dfe-3f91-4909-a717-9e0be1915f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:39.637918 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f82dfe-3f91-4909-a717-9e0be1915f97 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:39.644499 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:39.648917 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f82dfe-3f91-4909-a717-9e0be1915f97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:39.653473 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6022a3bd-477f-454f-93ed-0f7e75a6f17c None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:39.656716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-f7f82dfe-3f91-4909-a717-9e0be1915f97 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 21:02:39.656716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:39.656716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:39.656716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:39.656716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:39.656716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:39.656716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:39.656716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:39.656716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:39.656716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:39.656716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:39.656716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:39.656716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:39.656716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:39.656716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:39.656716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:39.657733 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:39.657733 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:39.657733 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:39.657733 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:39.657733 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:39.657733 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:39.657733 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:39.657733 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:39.657733 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:39.657733 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:39.657733 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:39.657733 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:39.657733 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:39.657733 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:39.657733 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:39.657733 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:39.658525 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:39.658525 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:39.658525 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:39.658525 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:39.658525 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:39.658525 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:39.658525 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:39.658525 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:39.658525 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:39.658525 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:39.658525 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 21:02:39.658525 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:02:39.658525 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 161/323] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:02:39.659634 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9555a382-27c4-4514-b9dc-4305f6a85833 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:39.659916 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9555a382-27c4-4514-b9dc-4305f6a85833 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:39.660190 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9555a382-27c4-4514-b9dc-4305f6a85833 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:39.661615 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 162/324] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:39.664132 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9555a382-27c4-4514-b9dc-4305f6a85833 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:39.688428 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd5e35-ab1b-4deb-8dc5-7bf671ec7669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:39.689265 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd5e35-ab1b-4deb-8dc5-7bf671ec7669 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:39.689545 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd5e35-ab1b-4deb-8dc5-7bf671ec7669 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:39.689815 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd5e35-ab1b-4deb-8dc5-7bf671ec7669 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:39.690983 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd5e35-ab1b-4deb-8dc5-7bf671ec7669 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:39.692433 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd5e35-ab1b-4deb-8dc5-7bf671ec7669 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:39.694075 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 163/325] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:02:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:39.695403 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec446767-e6b6-4c1e-bdca-267c8677d286 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:39.695689 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec446767-e6b6-4c1e-bdca-267c8677d286 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:39.695962 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec446767-e6b6-4c1e-bdca-267c8677d286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:39.712433 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-ec446767-e6b6-4c1e-bdca-267c8677d286 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:39.713674 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-ec446767-e6b6-4c1e-bdca-267c8677d286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:39.765149 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:39.778154 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9555a382-27c4-4514-b9dc-4305f6a85833 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:39.780892 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 164/326] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:39.783597 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec446767-e6b6-4c1e-bdca-267c8677d286 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:39.817955 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52b80f-23e2-4660-a4d0-136c89e6fd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:39.818619 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52b80f-23e2-4660-a4d0-136c89e6fd80 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:39.818894 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52b80f-23e2-4660-a4d0-136c89e6fd80 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:39.819166 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52b80f-23e2-4660-a4d0-136c89e6fd80 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:39.819916 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de52b80f-23e2-4660-a4d0-136c89e6fd80 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:39.820221 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de52b80f-23e2-4660-a4d0-136c89e6fd80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:39.821632 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de52b80f-23e2-4660-a4d0-136c89e6fd80 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:39.830640 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 163/327] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 27 21:02:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:39.831450 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52b80f-23e2-4660-a4d0-136c89e6fd80 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:39.877133 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:39.887060 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec446767-e6b6-4c1e-bdca-267c8677d286 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:39.921371 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1605cb9f-540a-450c-9a65-0ed2fffbdb54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:39.922041 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605cb9f-540a-450c-9a65-0ed2fffbdb54 None admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:39.922318 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605cb9f-540a-450c-9a65-0ed2fffbdb54 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:39.922589 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605cb9f-540a-450c-9a65-0ed2fffbdb54 None admin] PATH_INFO: `/v3/groups` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:39.923275 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1605cb9f-540a-450c-9a65-0ed2fffbdb54 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:39.924722 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1605cb9f-540a-450c-9a65-0ed2fffbdb54 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:39.932417 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52b80f-23e2-4660-a4d0-136c89e6fd80 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:39.939399 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 165/328] 10.222.0.13 () {60 vars in 1134 bytes} [Mon Jun 27 21:02:39 2022] POST /identity/v3/groups => generated 226 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:02:39.940214 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1605cb9f-540a-450c-9a65-0ed2fffbdb54 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:39.972307 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd93b02-44d7-4e1a-88e4-1a124ab1e3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:39.973163 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd93b02-44d7-4e1a-88e4-1a124ab1e3c7 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:39.973439 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd93b02-44d7-4e1a-88e4-1a124ab1e3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:39.973708 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd93b02-44d7-4e1a-88e4-1a124ab1e3c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:39.987371 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd93b02-44d7-4e1a-88e4-1a124ab1e3c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:39.995895 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-1cd93b02-44d7-4e1a-88e4-1a124ab1e3c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:39.995895 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:39.995895 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:39.995895 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:39.995895 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:39.995895 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:39.995895 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:39.995895 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:39.995895 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:39.995895 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:39.995895 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:39.995895 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:39.995895 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:39.995895 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:39.995895 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:39.995895 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:39.996874 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:39.996874 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:39.996874 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:39.996874 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:39.996874 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:39.996874 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:39.996874 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:39.996874 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:39.996874 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:39.996874 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:39.996874 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:39.996874 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:39.996874 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:39.996874 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:39.996874 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:39.996874 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:39.997724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:39.997724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:39.997724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:39.997724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:39.997724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:39.997724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:39.997724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:39.997724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:39.997724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:39.997724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:39.997724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:39.997724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:02:39.997724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 164/329] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:40.004948 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd93b02-44d7-4e1a-88e4-1a124ab1e3c7 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:40.022199 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:40.089564 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1605cb9f-540a-450c-9a65-0ed2fffbdb54 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:40.128053 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd93b02-44d7-4e1a-88e4-1a124ab1e3c7 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:40.139910 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd7d67-c9d3-47aa-b7df-58d9b9a3f6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:40.140815 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd7d67-c9d3-47aa-b7df-58d9b9a3f6e0 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:40.141128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd7d67-c9d3-47aa-b7df-58d9b9a3f6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:40.141400 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd7d67-c9d3-47aa-b7df-58d9b9a3f6e0 None admin] PATH_INFO: `/v3/users/8bed4e30d52b477abdc9fe06ff367c61` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:40.142716 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbd7d67-c9d3-47aa-b7df-58d9b9a3f6e0 None admin] RBAC: Authorizing `identity:get_user(user_id=8bed4e30d52b477abdc9fe06ff367c61)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:40.144315 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbd7d67-c9d3-47aa-b7df-58d9b9a3f6e0 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:40.147092 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 166/330] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:39 2022] GET /identity/v3/users/8bed4e30d52b477abdc9fe06ff367c61 => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:40.152770 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd7d67-c9d3-47aa-b7df-58d9b9a3f6e0 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:40.172906 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e43e5c-ea05-4cba-9d1a-36ec7a6af6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:40.173960 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e43e5c-ea05-4cba-9d1a-36ec7a6af6f4 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:40.174373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e43e5c-ea05-4cba-9d1a-36ec7a6af6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:40.174758 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e43e5c-ea05-4cba-9d1a-36ec7a6af6f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:40.176038 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e43e5c-ea05-4cba-9d1a-36ec7a6af6f4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:40.176540 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e43e5c-ea05-4cba-9d1a-36ec7a6af6f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:40.178992 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e43e5c-ea05-4cba-9d1a-36ec7a6af6f4 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:40.183478 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:40.194946 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 165/331] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 27 21:02:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:40.201202 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e43e5c-ea05-4cba-9d1a-36ec7a6af6f4 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:40.304079 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:40.322516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd7d67-c9d3-47aa-b7df-58d9b9a3f6e0 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:40.347668 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e43e5c-ea05-4cba-9d1a-36ec7a6af6f4 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:40.366206 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0151971f-47a9-4382-9238-3d3b77a36632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:40.368224 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0151971f-47a9-4382-9238-3d3b77a36632 None admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:40.368659 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0151971f-47a9-4382-9238-3d3b77a36632 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:40.369359 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0151971f-47a9-4382-9238-3d3b77a36632 None admin] PATH_INFO: `/v3/system/users/8bed4e30d52b477abdc9fe06ff367c61/roles/df30c7d320e947eaabb270652cb4afa3` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:40.371589 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0151971f-47a9-4382-9238-3d3b77a36632 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8bed4e30d52b477abdc9fe06ff367c61, role_id=df30c7d320e947eaabb270652cb4afa3)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:40.373182 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0151971f-47a9-4382-9238-3d3b77a36632 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:40.383691 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 167/332] 10.222.0.13 () {58 vars in 1334 bytes} [Mon Jun 27 21:02:40 2022] PUT /identity/v3/system/users/8bed4e30d52b477abdc9fe06ff367c61/roles/df30c7d320e947eaabb270652cb4afa3 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:02:40.393202 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d8e74-9ff7-47aa-9677-f72bba567d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:40.394274 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d8e74-9ff7-47aa-9677-f72bba567d2f None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:40.394553 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d8e74-9ff7-47aa-9677-f72bba567d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:40.394820 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d8e74-9ff7-47aa-9677-f72bba567d2f None admin] PATH_INFO: `/v3/users/09041e7213a84b91a459bef462360d2f` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:40.396108 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5d8e74-9ff7-47aa-9677-f72bba567d2f None admin] RBAC: Authorizing `identity:get_user(user_id=09041e7213a84b91a459bef462360d2f)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:40.397644 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5d8e74-9ff7-47aa-9677-f72bba567d2f None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:40.399413 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 166/333] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:40 2022] GET /identity/v3/users/09041e7213a84b91a459bef462360d2f => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:40.408311 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0151971f-47a9-4382-9238-3d3b77a36632 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:40.531995 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:40.564449 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0151971f-47a9-4382-9238-3d3b77a36632 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:40.615917 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf908c2-a121-47dc-951a-7c9688294993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:40.616593 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf908c2-a121-47dc-951a-7c9688294993 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:40.616873 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf908c2-a121-47dc-951a-7c9688294993 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:40.617219 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf908c2-a121-47dc-951a-7c9688294993 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:40.618314 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf908c2-a121-47dc-951a-7c9688294993 None admin] RBAC: Adding query filter params (role.id=e4414bde4df44de8aacb36c585a93e07, user.id=09041e7213a84b91a459bef462360d2f, scope.system=all) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:40.618624 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf908c2-a121-47dc-951a-7c9688294993 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:40.620195 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf908c2-a121-47dc-951a-7c9688294993 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:40.634950 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 168/334] 10.222.0.13 () {56 vars in 1308 bytes} [Mon Jun 27 21:02:40 2022] GET /identity/v3/role_assignments?user.id=09041e7213a84b91a459bef462360d2f&scope.system=all&role.id=e4414bde4df44de8aacb36c585a93e07 => generated 510 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:42.562400 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827fc85-e7b2-4661-a3b1-d6f4846be9a5 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:42.563490 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827fc85-e7b2-4661-a3b1-d6f4846be9a5 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:42.563989 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2827fc85-e7b2-4661-a3b1-d6f4846be9a5 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:42.565907 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 167/335] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:42.575807 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc027887-84d3-4dc0-8e6f-81b31670f565 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:42.576762 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc027887-84d3-4dc0-8e6f-81b31670f565 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:42.577328 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc027887-84d3-4dc0-8e6f-81b31670f565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:42.596411 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-cc027887-84d3-4dc0-8e6f-81b31670f565 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:42.598023 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-cc027887-84d3-4dc0-8e6f-81b31670f565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:42.659963 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc649192-0b67-4a2e-b6b4-512ca7afb7cd None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:42.660598 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc649192-0b67-4a2e-b6b4-512ca7afb7cd None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:42.661117 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc649192-0b67-4a2e-b6b4-512ca7afb7cd None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:42.662720 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 168/336] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:02:42.677501 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949a8f5-cc6b-4034-990e-78d9e1e95107 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:42.678134 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949a8f5-cc6b-4034-990e-78d9e1e95107 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:42.678611 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949a8f5-cc6b-4034-990e-78d9e1e95107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:42.706882 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-e949a8f5-cc6b-4034-990e-78d9e1e95107 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:42.707483 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 169/337] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:02:42.708996 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-e949a8f5-cc6b-4034-990e-78d9e1e95107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:42.786821 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 169/338] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:02:42.807275 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc027887-84d3-4dc0-8e6f-81b31670f565 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:42.818069 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e949a8f5-cc6b-4034-990e-78d9e1e95107 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:42.948447 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:42.972829 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:43.009728 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc027887-84d3-4dc0-8e6f-81b31670f565 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:43.028781 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e949a8f5-cc6b-4034-990e-78d9e1e95107 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:43.086384 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f321bb54-bcb7-49fe-b5e8-23af173780c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:43.087542 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321bb54-bcb7-49fe-b5e8-23af173780c5 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:43.087960 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321bb54-bcb7-49fe-b5e8-23af173780c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:43.088377 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321bb54-bcb7-49fe-b5e8-23af173780c5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:43.088559 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed86d6-cd74-496a-9f4f-bc57adcc35db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:43.089881 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed86d6-cd74-496a-9f4f-bc57adcc35db None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:43.090231 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed86d6-cd74-496a-9f4f-bc57adcc35db None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:43.090587 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed86d6-cd74-496a-9f4f-bc57adcc35db None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:43.103502 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f321bb54-bcb7-49fe-b5e8-23af173780c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:43.112399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-f321bb54-bcb7-49fe-b5e8-23af173780c5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:43.112399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:43.112399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:43.112399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:43.112399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:43.112399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:43.112399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:43.112399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:43.112399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:43.112399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:43.112399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:43.112399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:43.112399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:43.112399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:43.112399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:43.112399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:43.113558 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:43.113558 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:43.113558 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:43.113558 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:43.113558 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:43.113558 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:43.113558 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:43.113558 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:43.113558 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:43.113558 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:43.113558 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:43.113558 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:43.113558 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:43.113558 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:43.113558 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:43.113558 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:43.114570 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:43.114570 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:43.114570 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:43.114570 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:43.114570 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:43.114570 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:43.114570 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:43.114570 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:43.114570 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:43.114570 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:43.114570 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:43.114570 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:02:43.117035 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 170/339] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:43.118738 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d41e73-0d96-4d71-8909-3817d541772c None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:43.119179 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d41e73-0d96-4d71-8909-3817d541772c None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:43.119611 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d41e73-0d96-4d71-8909-3817d541772c None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:43.120624 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed86d6-cd74-496a-9f4f-bc57adcc35db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:43.121816 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 171/340] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:43.127192 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7c4a2-b090-47a1-9c60-1b40a74c15ee None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:43.127646 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7c4a2-b090-47a1-9c60-1b40a74c15ee None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:43.128072 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7c4a2-b090-47a1-9c60-1b40a74c15ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:43.133562 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-97ed86d6-cd74-496a-9f4f-bc57adcc35db None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:02:43.133562 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:43.133562 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:43.133562 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:43.133562 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:43.133562 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:43.133562 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:43.133562 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:43.133562 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:43.133562 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:43.133562 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:43.133562 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:43.133562 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:43.133562 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:43.133562 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:43.133562 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:43.134851 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:43.134851 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:43.134851 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:43.134851 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:43.134851 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:43.134851 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:43.134851 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:43.134851 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:43.134851 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:43.134851 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:43.134851 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:43.134851 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:43.134851 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:43.134851 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:43.134851 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:43.134851 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:43.135765 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:43.135765 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:43.135765 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:43.135765 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:43.135765 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:43.135765 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:43.135765 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:43.135765 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:43.135765 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:43.135765 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:43.135765 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:02:43.135765 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:02:43.135765 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 170/341] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 27 21:02:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:43.136464 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed86d6-cd74-496a-9f4f-bc57adcc35db None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:43.149924 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-25f7c4a2-b090-47a1-9c60-1b40a74c15ee None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:43.151445 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-25f7c4a2-b090-47a1-9c60-1b40a74c15ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:43.225904 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 172/342] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:02:43.227380 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7c4a2-b090-47a1-9c60-1b40a74c15ee None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:43.298059 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:43.343029 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7c4a2-b090-47a1-9c60-1b40a74c15ee None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:43.345160 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed86d6-cd74-496a-9f4f-bc57adcc35db None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:43.405515 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778169b3-3d7d-40c8-baf3-fe4b5f28fcbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:43.406208 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778169b3-3d7d-40c8-baf3-fe4b5f28fcbd None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:43.406490 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778169b3-3d7d-40c8-baf3-fe4b5f28fcbd None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:43.406768 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778169b3-3d7d-40c8-baf3-fe4b5f28fcbd None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:43.407537 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778169b3-3d7d-40c8-baf3-fe4b5f28fcbd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:43.407848 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778169b3-3d7d-40c8-baf3-fe4b5f28fcbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:43.409288 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778169b3-3d7d-40c8-baf3-fe4b5f28fcbd None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:43.413018 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:43.420892 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8f91a-9f3b-43d1-a479-87e04df46fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:43.421603 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8f91a-9f3b-43d1-a479-87e04df46fdd None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:43.421890 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8f91a-9f3b-43d1-a479-87e04df46fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:43.422163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8f91a-9f3b-43d1-a479-87e04df46fdd None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:43.422928 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8f91a-9f3b-43d1-a479-87e04df46fdd None admin] RBAC: Adding query filter params (name=member) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:43.423239 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8f91a-9f3b-43d1-a479-87e04df46fdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:43.424646 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8f91a-9f3b-43d1-a479-87e04df46fdd None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:43.436862 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 173/343] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:02:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:43.437579 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778169b3-3d7d-40c8-baf3-fe4b5f28fcbd None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:43.443390 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 171/344] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 27 21:02:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:43.444239 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8f91a-9f3b-43d1-a479-87e04df46fdd None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:43.619113 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8f91a-9f3b-43d1-a479-87e04df46fdd None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:43.654712 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778169b3-3d7d-40c8-baf3-fe4b5f28fcbd None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:43.694716 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b81e47-6a52-4f97-bb2d-28b3e0803a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:43.695441 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b81e47-6a52-4f97-bb2d-28b3e0803a66 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:43.695724 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b81e47-6a52-4f97-bb2d-28b3e0803a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:43.696023 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b81e47-6a52-4f97-bb2d-28b3e0803a66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:43.701952 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:43.727298 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b09813f-888a-495f-a9c3-f451e33defa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:43.728123 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09813f-888a-495f-a9c3-f451e33defa0 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:43.728399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09813f-888a-495f-a9c3-f451e33defa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:43.728670 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09813f-888a-495f-a9c3-f451e33defa0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:43.741735 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b09813f-888a-495f-a9c3-f451e33defa0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:43.755718 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-8b09813f-888a-495f-a9c3-f451e33defa0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 21:02:43.755718 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:43.755718 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:43.755718 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:43.755718 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:43.755718 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:43.755718 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:43.755718 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:43.755718 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:43.755718 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:43.755718 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:43.755718 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:43.755718 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:43.755718 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:43.755718 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:43.755718 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:43.756695 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:43.756695 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:43.756695 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:43.756695 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:43.756695 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:43.756695 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:43.756695 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:43.756695 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:43.756695 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:43.756695 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:43.756695 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:43.756695 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:43.756695 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:43.756695 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:43.756695 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:43.756695 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:43.757534 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:43.757534 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:43.757534 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:43.757534 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:43.757534 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:43.757534 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:43.757534 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:43.757534 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:43.757534 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:43.757534 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:43.757534 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 27 21:02:43.757534 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:02:43.760045 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 174/345] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:43.761002 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b09813f-888a-495f-a9c3-f451e33defa0 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:43.823343 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:43.838527 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b81e47-6a52-4f97-bb2d-28b3e0803a66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:43.840065 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b81e47-6a52-4f97-bb2d-28b3e0803a66 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:43.930258 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b09813f-888a-495f-a9c3-f451e33defa0 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:43.982860 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d35b6-a4bf-4d84-8c4a-2d54bef8b75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:43.983523 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d35b6-a4bf-4d84-8c4a-2d54bef8b75e None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:43.983806 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d35b6-a4bf-4d84-8c4a-2d54bef8b75e None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:43.984210 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d35b6-a4bf-4d84-8c4a-2d54bef8b75e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:43.987384 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 172/346] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:02:43.989131 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b81e47-6a52-4f97-bb2d-28b3e0803a66 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:44.073944 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:44.100780 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5d35b6-a4bf-4d84-8c4a-2d54bef8b75e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:44.104359 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5d35b6-a4bf-4d84-8c4a-2d54bef8b75e None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:44.114235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:44.125601 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b81e47-6a52-4f97-bb2d-28b3e0803a66 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:44.180625 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a2afb-b818-4715-b239-b16f383a49e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:44.181631 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a2afb-b818-4715-b239-b16f383a49e7 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:44.182113 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a2afb-b818-4715-b239-b16f383a49e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:44.182550 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a2afb-b818-4715-b239-b16f383a49e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:44.183876 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3a2afb-b818-4715-b239-b16f383a49e7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:44.184359 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3a2afb-b818-4715-b239-b16f383a49e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:44.186092 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3a2afb-b818-4715-b239-b16f383a49e7 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:44.197217 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 173/347] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 27 21:02:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:44.200181 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1010bb7-724a-4270-9601-f312a718106d None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:44.200644 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1010bb7-724a-4270-9601-f312a718106d None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:44.201114 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1010bb7-724a-4270-9601-f312a718106d None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:44.203015 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 174/348] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:44.204001 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1010bb7-724a-4270-9601-f312a718106d None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:44.253255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 175/349] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:02:44.254295 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d35b6-a4bf-4d84-8c4a-2d54bef8b75e None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:44.343656 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1010bb7-724a-4270-9601-f312a718106d None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:44.350478 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:44.376164 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d35b6-a4bf-4d84-8c4a-2d54bef8b75e None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:44.391428 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf71ce8-4e8b-41df-a974-aeca9326f783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:44.392770 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf71ce8-4e8b-41df-a974-aeca9326f783 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:44.394859 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf71ce8-4e8b-41df-a974-aeca9326f783 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:44.394859 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf71ce8-4e8b-41df-a974-aeca9326f783 None admin] PATH_INFO: `/v3/projects/04d09da302f44fefbe5b4502010ec050` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:44.394859 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf71ce8-4e8b-41df-a974-aeca9326f783 None admin] RBAC: Authorizing `identity:get_project(project_id=04d09da302f44fefbe5b4502010ec050)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:44.398192 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf71ce8-4e8b-41df-a974-aeca9326f783 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:44.398192 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 175/350] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:44 2022] GET /identity/v3/projects/04d09da302f44fefbe5b4502010ec050 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:44.399174 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec07786-e2ad-4870-a037-177fde53943c None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:44.399697 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec07786-e2ad-4870-a037-177fde53943c None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:44.400182 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec07786-e2ad-4870-a037-177fde53943c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:44.419645 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-eec07786-e2ad-4870-a037-177fde53943c None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:44.420654 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a97d96e-0edd-4dca-a301-d713866ed3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:44.421490 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-eec07786-e2ad-4870-a037-177fde53943c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:44.421778 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a97d96e-0edd-4dca-a301-d713866ed3ad None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:44.422172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a97d96e-0edd-4dca-a301-d713866ed3ad None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:44.422450 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a97d96e-0edd-4dca-a301-d713866ed3ad None admin] PATH_INFO: `/v3/users/8bed4e30d52b477abdc9fe06ff367c61` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:44.423887 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a97d96e-0edd-4dca-a301-d713866ed3ad None admin] RBAC: Authorizing `identity:get_user(user_id=8bed4e30d52b477abdc9fe06ff367c61)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:44.426733 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a97d96e-0edd-4dca-a301-d713866ed3ad None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:44.428872 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 176/351] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:02:44 2022] GET /identity/v3/users/8bed4e30d52b477abdc9fe06ff367c61 => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:44.429751 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a97d96e-0edd-4dca-a301-d713866ed3ad None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:44.447195 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:44.516737 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 176/352] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:44.518467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1fcf75-99a3-4b4c-a563-8707d91465f3 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:44.518981 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1fcf75-99a3-4b4c-a563-8707d91465f3 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:44.519453 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1fcf75-99a3-4b4c-a563-8707d91465f3 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:44.520965 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 177/353] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:44.521924 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1fcf75-99a3-4b4c-a563-8707d91465f3 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:44.578968 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a97d96e-0edd-4dca-a301-d713866ed3ad None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:44.629806 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7babbad4-85a2-464b-82d9-723e8d919f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:44.630717 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7babbad4-85a2-464b-82d9-723e8d919f90 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:44.630995 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7babbad4-85a2-464b-82d9-723e8d919f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:44.631267 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7babbad4-85a2-464b-82d9-723e8d919f90 None admin] PATH_INFO: `/v3/projects/04d09da302f44fefbe5b4502010ec050` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:44.632502 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7babbad4-85a2-464b-82d9-723e8d919f90 None admin] RBAC: Authorizing `identity:get_project(project_id=04d09da302f44fefbe5b4502010ec050)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:44.634049 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7babbad4-85a2-464b-82d9-723e8d919f90 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:44.636197 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 177/354] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:44 2022] GET /identity/v3/projects/04d09da302f44fefbe5b4502010ec050 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:44.637365 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7babbad4-85a2-464b-82d9-723e8d919f90 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:44.663742 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:44.678668 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1fcf75-99a3-4b4c-a563-8707d91465f3 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:44.742773 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9d7cd1-5ded-42bb-8cd0-096c64928fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:44.743109 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d7cd1-5ded-42bb-8cd0-096c64928fb9 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:44.743383 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d7cd1-5ded-42bb-8cd0-096c64928fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:44.743658 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d7cd1-5ded-42bb-8cd0-096c64928fb9 None admin] PATH_INFO: `/v3/groups/11aca19a34c34bc28cf9d777aef5f6ab` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:44.752461 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9d7cd1-5ded-42bb-8cd0-096c64928fb9 None admin] RBAC: Authorizing `identity:get_group(group_id=11aca19a34c34bc28cf9d777aef5f6ab)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:44.754100 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9d7cd1-5ded-42bb-8cd0-096c64928fb9 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:44.761059 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 178/355] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Jun 27 21:02:44 2022] GET /identity/v3/groups/11aca19a34c34bc28cf9d777aef5f6ab => generated 226 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:44.762842 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a21b5c-8b43-4999-a1eb-8f942a3b2891 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:44.763206 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a21b5c-8b43-4999-a1eb-8f942a3b2891 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:44.763561 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a21b5c-8b43-4999-a1eb-8f942a3b2891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:44.789057 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-e6a21b5c-8b43-4999-a1eb-8f942a3b2891 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:44.790196 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-e6a21b5c-8b43-4999-a1eb-8f942a3b2891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:44.802545 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7babbad4-85a2-464b-82d9-723e8d919f90 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:44.803074 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:44.856551 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5871f81-814e-452c-9c80-252d3138d5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:44.857385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5871f81-814e-452c-9c80-252d3138d5c2 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:44.857664 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5871f81-814e-452c-9c80-252d3138d5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:44.857940 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5871f81-814e-452c-9c80-252d3138d5c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:44.858873 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5871f81-814e-452c-9c80-252d3138d5c2 None admin] RBAC: Adding query filter params (role.id=df30c7d320e947eaabb270652cb4afa3, user.id=8bed4e30d52b477abdc9fe06ff367c61, scope.system=all) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:44.859184 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5871f81-814e-452c-9c80-252d3138d5c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:44.860642 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5871f81-814e-452c-9c80-252d3138d5c2 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:44.880004 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 178/356] 10.222.0.13 () {56 vars in 1308 bytes} [Mon Jun 27 21:02:44 2022] GET /identity/v3/role_assignments?user.id=8bed4e30d52b477abdc9fe06ff367c61&scope.system=all&role.id=df30c7d320e947eaabb270652cb4afa3 => generated 510 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:44.880004 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5871f81-814e-452c-9c80-252d3138d5c2 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:44.889020 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 179/357] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:44.890458 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a21b5c-8b43-4999-a1eb-8f942a3b2891 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:44.994234 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:45.037581 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5871f81-814e-452c-9c80-252d3138d5c2 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:45.060023 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a21b5c-8b43-4999-a1eb-8f942a3b2891 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:45.093376 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34550cae-c4fa-4460-b14a-9c7fd974d71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:45.094598 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34550cae-c4fa-4460-b14a-9c7fd974d71b None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:45.095128 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34550cae-c4fa-4460-b14a-9c7fd974d71b None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:45.095473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34550cae-c4fa-4460-b14a-9c7fd974d71b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:45.113990 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34550cae-c4fa-4460-b14a-9c7fd974d71b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:45.122448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-34550cae-c4fa-4460-b14a-9c7fd974d71b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:45.122448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:45.122448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:45.122448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:45.122448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:45.122448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:45.122448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:45.122448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:45.122448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:45.122448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:45.122448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:45.122448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:45.122448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:45.122448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:45.122448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:45.122448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:45.123577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:45.123577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:45.123577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:45.123577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:45.123577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:45.123577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:45.123577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:45.123577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:45.123577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:45.123577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:45.123577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:45.123577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:45.123577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:45.123577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:45.123577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:45.123577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:45.124379 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:45.124379 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:45.124379 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:45.124379 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:45.124379 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:45.124379 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:45.124379 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:45.124379 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:45.124379 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:45.124379 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:45.124379 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:45.124379 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:02:45.125131 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0309493-9e49-41f3-bf35-def20b9924da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:45.126406 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0309493-9e49-41f3-bf35-def20b9924da None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:45.126943 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0309493-9e49-41f3-bf35-def20b9924da None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:45.127528 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0309493-9e49-41f3-bf35-def20b9924da None admin] PATH_INFO: `/v3/groups/11aca19a34c34bc28cf9d777aef5f6ab` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:45.130525 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0309493-9e49-41f3-bf35-def20b9924da None admin] RBAC: Authorizing `identity:get_group(group_id=11aca19a34c34bc28cf9d777aef5f6ab)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:45.131510 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0309493-9e49-41f3-bf35-def20b9924da None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:45.131933 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 179/358] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:45.132993 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34550cae-c4fa-4460-b14a-9c7fd974d71b None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:45.133561 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 180/359] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Jun 27 21:02:44 2022] GET /identity/v3/groups/11aca19a34c34bc28cf9d777aef5f6ab => generated 226 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:45.134661 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0309493-9e49-41f3-bf35-def20b9924da None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:45.158429 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:45.261660 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0309493-9e49-41f3-bf35-def20b9924da None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:45.265868 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34550cae-c4fa-4460-b14a-9c7fd974d71b None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:45.311931 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec62e0f2-9fef-46b0-b18d-3af6be0644d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:45.312628 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec62e0f2-9fef-46b0-b18d-3af6be0644d2 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:45.312998 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec62e0f2-9fef-46b0-b18d-3af6be0644d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:45.313266 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec62e0f2-9fef-46b0-b18d-3af6be0644d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:45.314418 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec62e0f2-9fef-46b0-b18d-3af6be0644d2 None admin] RBAC: Adding query filter params (group.id=11aca19a34c34bc28cf9d777aef5f6ab, role.id=5834784bf54d4806b88e32e0fb8e18fe, scope.project.id=04d09da302f44fefbe5b4502010ec050) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:45.314549 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76806e92-6450-43af-90d7-366253707512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:45.314643 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec62e0f2-9fef-46b0-b18d-3af6be0644d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:45.315569 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76806e92-6450-43af-90d7-366253707512 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:45.317864 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76806e92-6450-43af-90d7-366253707512 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:45.317982 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec62e0f2-9fef-46b0-b18d-3af6be0644d2 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:45.318059 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76806e92-6450-43af-90d7-366253707512 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:45.320329 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:45.326499 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 180/360] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Jun 27 21:02:45 2022] GET /identity/v3/role_assignments?group.id=11aca19a34c34bc28cf9d777aef5f6ab&scope.project.id=04d09da302f44fefbe5b4502010ec050&role.id=5834784bf54d4806b88e32e0fb8e18fe => generated 260 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:45.328980 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4366fb-5e42-4817-a41c-b518dcfe52c8 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:45.329508 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4366fb-5e42-4817-a41c-b518dcfe52c8 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:45.330078 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4366fb-5e42-4817-a41c-b518dcfe52c8 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:45.332107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76806e92-6450-43af-90d7-366253707512 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:45.332357 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 181/361] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:45.333793 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4366fb-5e42-4817-a41c-b518dcfe52c8 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:45.341516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-76806e92-6450-43af-90d7-366253707512 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:02:45.341516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:45.341516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:45.341516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:45.341516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:45.341516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:45.341516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:45.341516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:45.341516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:45.341516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:45.341516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:45.341516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:45.341516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:45.341516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:45.341516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:45.341516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:45.342537 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:45.342537 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:45.342537 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:45.342537 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:45.342537 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:45.342537 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:45.342537 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:45.342537 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:45.342537 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:45.342537 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:45.342537 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:45.342537 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:45.342537 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:45.342537 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:45.342537 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:45.342537 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:45.343445 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:45.343445 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:45.343445 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:45.343445 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:45.343445 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:45.343445 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:45.343445 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:45.343445 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:45.343445 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:45.343445 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:45.343445 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:02:45.343445 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:02:45.344156 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 181/362] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 27 21:02:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:45.345621 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76806e92-6450-43af-90d7-366253707512 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:45.448364 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:45.488067 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76806e92-6450-43af-90d7-366253707512 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:45.489547 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4366fb-5e42-4817-a41c-b518dcfe52c8 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:45.537645 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33950f5b-14c7-4a53-89c8-8f75e9806916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:45.538710 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33950f5b-14c7-4a53-89c8-8f75e9806916 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:45.539063 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33950f5b-14c7-4a53-89c8-8f75e9806916 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:45.539410 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33950f5b-14c7-4a53-89c8-8f75e9806916 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:45.540249 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33950f5b-14c7-4a53-89c8-8f75e9806916 None admin] RBAC: Adding query filter params (name=member) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:45.540626 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33950f5b-14c7-4a53-89c8-8f75e9806916 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:45.541139 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75820305-7391-41de-833b-13b4434f94b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:45.542021 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75820305-7391-41de-833b-13b4434f94b7 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:45.542269 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33950f5b-14c7-4a53-89c8-8f75e9806916 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:45.542881 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75820305-7391-41de-833b-13b4434f94b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:45.543567 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75820305-7391-41de-833b-13b4434f94b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:45.544635 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75820305-7391-41de-833b-13b4434f94b7 None admin] RBAC: Adding query filter params (group.id=11aca19a34c34bc28cf9d777aef5f6ab, role.id=e4414bde4df44de8aacb36c585a93e07, scope.project.id=04d09da302f44fefbe5b4502010ec050) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:45.545203 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75820305-7391-41de-833b-13b4434f94b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:45.546890 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75820305-7391-41de-833b-13b4434f94b7 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:45.554347 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 182/363] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Jun 27 21:02:45 2022] GET /identity/v3/role_assignments?group.id=11aca19a34c34bc28cf9d777aef5f6ab&scope.project.id=04d09da302f44fefbe5b4502010ec050&role.id=e4414bde4df44de8aacb36c585a93e07 => generated 260 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:45.554739 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 182/364] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 27 21:02:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:45.555623 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33950f5b-14c7-4a53-89c8-8f75e9806916 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:45.556553 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8abeb9c-a505-426f-aa10-c776ffdd4f47 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:45.557118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8abeb9c-a505-426f-aa10-c776ffdd4f47 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:45.557611 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8abeb9c-a505-426f-aa10-c776ffdd4f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:45.579288 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-f8abeb9c-a505-426f-aa10-c776ffdd4f47 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:45.581056 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-f8abeb9c-a505-426f-aa10-c776ffdd4f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:45.642022 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:45.679282 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 183/365] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:45.682018 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8abeb9c-a505-426f-aa10-c776ffdd4f47 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:45.724280 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33950f5b-14c7-4a53-89c8-8f75e9806916 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:45.777878 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14d06fa-75af-4167-8b59-63f7fbc4fb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:45.777878 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d06fa-75af-4167-8b59-63f7fbc4fb20 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:45.777878 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d06fa-75af-4167-8b59-63f7fbc4fb20 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:45.778207 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d06fa-75af-4167-8b59-63f7fbc4fb20 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:45.778934 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14d06fa-75af-4167-8b59-63f7fbc4fb20 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:45.779243 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14d06fa-75af-4167-8b59-63f7fbc4fb20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:45.780778 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14d06fa-75af-4167-8b59-63f7fbc4fb20 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:45.789899 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 183/366] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:02:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:45.790677 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14d06fa-75af-4167-8b59-63f7fbc4fb20 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:45.802539 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:45.845361 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8abeb9c-a505-426f-aa10-c776ffdd4f47 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:45.883865 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9cdf6d-0dc5-4962-9e4d-35c90d44a28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:45.884631 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9cdf6d-0dc5-4962-9e4d-35c90d44a28c None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:45.884954 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9cdf6d-0dc5-4962-9e4d-35c90d44a28c None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:45.885220 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9cdf6d-0dc5-4962-9e4d-35c90d44a28c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:45.901677 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14d06fa-75af-4167-8b59-63f7fbc4fb20 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:45.938359 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:45.954184 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741f1988-b13e-4179-af46-357da8905a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:45.955283 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f1988-b13e-4179-af46-357da8905a74 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:45.955802 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f1988-b13e-4179-af46-357da8905a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:45.956092 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f1988-b13e-4179-af46-357da8905a74 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:45.965552 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9cdf6d-0dc5-4962-9e4d-35c90d44a28c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:45.967319 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9cdf6d-0dc5-4962-9e4d-35c90d44a28c None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:45.977552 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741f1988-b13e-4179-af46-357da8905a74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:45.988268 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-741f1988-b13e-4179-af46-357da8905a74 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:02:45.988268 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:45.988268 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:45.988268 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:45.988268 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:45.988268 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:45.988268 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:45.988268 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:45.988268 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:45.988268 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:45.988268 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:45.988268 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:45.988268 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:45.988268 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:45.988268 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:45.988268 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:45.989277 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:45.989277 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:45.989277 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:45.989277 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:45.989277 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:45.989277 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:45.989277 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:45.989277 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:45.989277 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:45.989277 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:45.989277 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:45.989277 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:45.989277 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:45.989277 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:45.989277 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:45.989277 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:45.990069 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:45.990069 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:45.990069 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:45.990069 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:45.990069 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:45.990069 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:45.990069 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:45.990069 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:45.990069 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:45.990069 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:45.990069 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:02:45.990069 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:02:45.990069 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 184/367] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 27 21:02:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:45.992317 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741f1988-b13e-4179-af46-357da8905a74 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:46.082497 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:46.133721 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 184/368] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:02:46.134844 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9cdf6d-0dc5-4962-9e4d-35c90d44a28c None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:46.155380 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741f1988-b13e-4179-af46-357da8905a74 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:46.200178 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c61f54-3c0c-4d42-b81d-606890c5dfc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:46.201272 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c61f54-3c0c-4d42-b81d-606890c5dfc7 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:46.201783 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c61f54-3c0c-4d42-b81d-606890c5dfc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:46.202277 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c61f54-3c0c-4d42-b81d-606890c5dfc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:46.264088 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:46.290666 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c61f54-3c0c-4d42-b81d-606890c5dfc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:46.292853 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c61f54-3c0c-4d42-b81d-606890c5dfc7 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:46.295387 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9cdf6d-0dc5-4962-9e4d-35c90d44a28c None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:46.341300 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9ba25b-62f2-490a-a425-ded13a8154fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:46.342480 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ba25b-62f2-490a-a425-ded13a8154fe None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:46.343041 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ba25b-62f2-490a-a425-ded13a8154fe None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:46.343562 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ba25b-62f2-490a-a425-ded13a8154fe None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:46.344856 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9ba25b-62f2-490a-a425-ded13a8154fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:46.345770 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9ba25b-62f2-490a-a425-ded13a8154fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:46.348062 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9ba25b-62f2-490a-a425-ded13a8154fe None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:46.366280 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 185/369] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:46.368159 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9ba25b-62f2-490a-a425-ded13a8154fe None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:46.403612 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:46.432962 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 185/370] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:02:46.432962 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c61f54-3c0c-4d42-b81d-606890c5dfc7 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:46.489060 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9ba25b-62f2-490a-a425-ded13a8154fe None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:46.523224 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:46.528262 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad285cf-e6e7-4763-91f9-a1550eb441c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:46.529531 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad285cf-e6e7-4763-91f9-a1550eb441c9 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:46.530012 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad285cf-e6e7-4763-91f9-a1550eb441c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:46.530504 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad285cf-e6e7-4763-91f9-a1550eb441c9 None admin] PATH_INFO: `/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:46.532122 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad285cf-e6e7-4763-91f9-a1550eb441c9 None admin] RBAC: Authorizing `identity:get_project(project_id=8fc48df8fb1f4de59d32fefd09a8ecae)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:46.533796 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad285cf-e6e7-4763-91f9-a1550eb441c9 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:46.535885 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 186/371] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:46 2022] GET /identity/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:46.537032 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad285cf-e6e7-4763-91f9-a1550eb441c9 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:46.540648 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c61f54-3c0c-4d42-b81d-606890c5dfc7 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:46.580584 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a1e76-1170-467b-83e0-0cbe59005c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:46.581326 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a1e76-1170-467b-83e0-0cbe59005c03 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:46.581608 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a1e76-1170-467b-83e0-0cbe59005c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:46.581882 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a1e76-1170-467b-83e0-0cbe59005c03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:46.648860 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:46.653592 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad285cf-e6e7-4763-91f9-a1550eb441c9 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:46.657555 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81a1e76-1170-467b-83e0-0cbe59005c03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:46.659058 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81a1e76-1170-467b-83e0-0cbe59005c03 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:46.724117 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a40fe15-87a3-489b-9d7e-0a442a02bd6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:46.724117 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40fe15-87a3-489b-9d7e-0a442a02bd6b None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:46.724117 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40fe15-87a3-489b-9d7e-0a442a02bd6b None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:46.724815 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40fe15-87a3-489b-9d7e-0a442a02bd6b None admin] PATH_INFO: `/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:46.726632 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a40fe15-87a3-489b-9d7e-0a442a02bd6b None admin] RBAC: Authorizing `identity:get_project(project_id=8fc48df8fb1f4de59d32fefd09a8ecae)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:46.728451 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a40fe15-87a3-489b-9d7e-0a442a02bd6b None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:46.730313 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 187/372] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:46 2022] GET /identity/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:46.731096 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a40fe15-87a3-489b-9d7e-0a442a02bd6b None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:46.774631 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:46.801080 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 186/373] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:02:46.802827 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a1e76-1170-467b-83e0-0cbe59005c03 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:46.876631 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a40fe15-87a3-489b-9d7e-0a442a02bd6b None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:46.921864 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e11199d-159b-403b-8229-a8122620ccad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:46.922999 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11199d-159b-403b-8229-a8122620ccad None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:46.923284 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11199d-159b-403b-8229-a8122620ccad None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:46.923560 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11199d-159b-403b-8229-a8122620ccad None admin] PATH_INFO: `/v3/groups/11aca19a34c34bc28cf9d777aef5f6ab` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:46.924943 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e11199d-159b-403b-8229-a8122620ccad None admin] RBAC: Authorizing `identity:get_group(group_id=11aca19a34c34bc28cf9d777aef5f6ab)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:46.926737 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e11199d-159b-403b-8229-a8122620ccad None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:46.928728 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 188/374] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Jun 27 21:02:46 2022] GET /identity/v3/groups/11aca19a34c34bc28cf9d777aef5f6ab => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:46.929560 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e11199d-159b-403b-8229-a8122620ccad None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:46.950388 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a1e76-1170-467b-83e0-0cbe59005c03 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:46.973371 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:46.998545 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ec469-ad46-42b7-8871-d71936609b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:46.999802 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ec469-ad46-42b7-8871-d71936609b1d None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:47.000281 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ec469-ad46-42b7-8871-d71936609b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:47.000750 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ec469-ad46-42b7-8871-d71936609b1d None admin] PATH_INFO: `/v3/groups/11aca19a34c34bc28cf9d777aef5f6ab` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:47.002259 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15ec469-ad46-42b7-8871-d71936609b1d None admin] RBAC: Authorizing `identity:get_group(group_id=11aca19a34c34bc28cf9d777aef5f6ab)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:47.004110 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15ec469-ad46-42b7-8871-d71936609b1d None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:47.011465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 187/375] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Jun 27 21:02:46 2022] GET /identity/v3/groups/11aca19a34c34bc28cf9d777aef5f6ab => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:47.012756 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ec469-ad46-42b7-8871-d71936609b1d None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:47.061262 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e11199d-159b-403b-8229-a8122620ccad None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:47.087710 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:47.106088 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca8b0ff-0f01-4dee-aff8-1a2d1d06a244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:47.107516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8b0ff-0f01-4dee-aff8-1a2d1d06a244 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:47.108079 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8b0ff-0f01-4dee-aff8-1a2d1d06a244 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:47.108639 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8b0ff-0f01-4dee-aff8-1a2d1d06a244 None admin] PATH_INFO: `/v3/projects/b6e5cf8b46f34e30a9b900f3cd1507c0` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:47.110382 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca8b0ff-0f01-4dee-aff8-1a2d1d06a244 None admin] RBAC: Authorizing `identity:get_project(project_id=b6e5cf8b46f34e30a9b900f3cd1507c0)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:47.112261 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca8b0ff-0f01-4dee-aff8-1a2d1d06a244 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:47.114488 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 189/376] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:46 2022] GET /identity/v3/projects/b6e5cf8b46f34e30a9b900f3cd1507c0 => generated 344 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:47.115978 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca8b0ff-0f01-4dee-aff8-1a2d1d06a244 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:47.145899 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ec469-ad46-42b7-8871-d71936609b1d None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:47.189043 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c0c0f4-d9f7-49c2-b8cb-44d0334a528d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:47.190155 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c0c0f4-d9f7-49c2-b8cb-44d0334a528d None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:47.190634 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c0c0f4-d9f7-49c2-b8cb-44d0334a528d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:47.191124 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c0c0f4-d9f7-49c2-b8cb-44d0334a528d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:47.192163 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c0c0f4-d9f7-49c2-b8cb-44d0334a528d None admin] RBAC: Adding query filter params (group.id=11aca19a34c34bc28cf9d777aef5f6ab, role.id=e4414bde4df44de8aacb36c585a93e07, scope.project.id=8fc48df8fb1f4de59d32fefd09a8ecae) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:47.192699 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c0c0f4-d9f7-49c2-b8cb-44d0334a528d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:47.194453 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c0c0f4-d9f7-49c2-b8cb-44d0334a528d None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:47.208529 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 188/377] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Jun 27 21:02:47 2022] GET /identity/v3/role_assignments?group.id=11aca19a34c34bc28cf9d777aef5f6ab&scope.project.id=8fc48df8fb1f4de59d32fefd09a8ecae&role.id=e4414bde4df44de8aacb36c585a93e07 => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:47.210029 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c0c0f4-d9f7-49c2-b8cb-44d0334a528d None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:47.246995 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca8b0ff-0f01-4dee-aff8-1a2d1d06a244 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:47.265387 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:47.294060 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac95249-d62b-4567-af03-1be231ff3b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:47.294860 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac95249-d62b-4567-af03-1be231ff3b1c None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:47.295214 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac95249-d62b-4567-af03-1be231ff3b1c None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:47.295496 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac95249-d62b-4567-af03-1be231ff3b1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:47.296544 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac95249-d62b-4567-af03-1be231ff3b1c None admin] RBAC: Adding query filter params (group.id=11aca19a34c34bc28cf9d777aef5f6ab, role.id=5834784bf54d4806b88e32e0fb8e18fe, scope.project.id=8fc48df8fb1f4de59d32fefd09a8ecae) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:47.300957 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac95249-d62b-4567-af03-1be231ff3b1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:47.300957 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac95249-d62b-4567-af03-1be231ff3b1c None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:47.305942 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 190/378] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Jun 27 21:02:47 2022] GET /identity/v3/role_assignments?group.id=11aca19a34c34bc28cf9d777aef5f6ab&scope.project.id=8fc48df8fb1f4de59d32fefd09a8ecae&role.id=5834784bf54d4806b88e32e0fb8e18fe => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:47.414619 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c0c0f4-d9f7-49c2-b8cb-44d0334a528d None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:47.446112 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:47.468673 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459e0031-7f1a-4fc5-a804-d1761e3f0054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:47.470229 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e0031-7f1a-4fc5-a804-d1761e3f0054 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:47.470792 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e0031-7f1a-4fc5-a804-d1761e3f0054 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:47.471281 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e0031-7f1a-4fc5-a804-d1761e3f0054 None admin] PATH_INFO: `/v3/groups/2c24848f4d824e949955fab72678075d` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:47.483063 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459e0031-7f1a-4fc5-a804-d1761e3f0054 None admin] RBAC: Authorizing `identity:get_group(group_id=2c24848f4d824e949955fab72678075d)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:47.485645 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459e0031-7f1a-4fc5-a804-d1761e3f0054 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:47.487777 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 189/379] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Jun 27 21:02:47 2022] GET /identity/v3/groups/2c24848f4d824e949955fab72678075d => generated 229 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:47.494060 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac95249-d62b-4567-af03-1be231ff3b1c None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:47.633299 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac95249-d62b-4567-af03-1be231ff3b1c None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:47.673302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff6299-6d35-4f78-9d06-15d267c7e7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:47.674564 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff6299-6d35-4f78-9d06-15d267c7e7c0 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:47.675135 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff6299-6d35-4f78-9d06-15d267c7e7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:47.675666 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff6299-6d35-4f78-9d06-15d267c7e7c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:47.676848 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eff6299-6d35-4f78-9d06-15d267c7e7c0 None admin] RBAC: Adding query filter params (group.id=2c24848f4d824e949955fab72678075d, role.id=d336dd7b746c4488b33289c00302e4d5, scope.project.id=b6e5cf8b46f34e30a9b900f3cd1507c0) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:47.677542 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eff6299-6d35-4f78-9d06-15d267c7e7c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:47.679823 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eff6299-6d35-4f78-9d06-15d267c7e7c0 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:47.687293 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 191/380] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Jun 27 21:02:47 2022] GET /identity/v3/role_assignments?group.id=2c24848f4d824e949955fab72678075d&scope.project.id=b6e5cf8b46f34e30a9b900f3cd1507c0&role.id=d336dd7b746c4488b33289c00302e4d5 => generated 260 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:48.029819 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3792c2b-624a-4700-a03d-e416e71b9692 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:48.030351 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3792c2b-624a-4700-a03d-e416e71b9692 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:48.030351 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3792c2b-624a-4700-a03d-e416e71b9692 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:48.032012 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 190/381] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:48.037653 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb145989-84a4-498e-9472-b380f583675f None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:48.038103 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb145989-84a4-498e-9472-b380f583675f None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:48.038205 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb145989-84a4-498e-9472-b380f583675f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:48.047442 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a9611-52d3-4858-a739-97a4cea71ace None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:48.047723 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a9611-52d3-4858-a739-97a4cea71ace None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:48.047996 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a9611-52d3-4858-a739-97a4cea71ace None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:48.049436 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 191/382] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:48.056232 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887237fc-26ef-463c-aaab-84b689d93575 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:48.056518 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887237fc-26ef-463c-aaab-84b689d93575 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:48.056790 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887237fc-26ef-463c-aaab-84b689d93575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:48.065733 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-eb145989-84a4-498e-9472-b380f583675f None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:48.067244 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-eb145989-84a4-498e-9472-b380f583675f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:48.085712 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-887237fc-26ef-463c-aaab-84b689d93575 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:48.087094 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-887237fc-26ef-463c-aaab-84b689d93575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:48.095182 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:48.199135 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 192/383] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:48.233965 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb145989-84a4-498e-9472-b380f583675f None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:48.234380 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 192/384] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:02:48.261612 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887237fc-26ef-463c-aaab-84b689d93575 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:48.366529 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb145989-84a4-498e-9472-b380f583675f None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:48.430604 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff139757-94d0-4120-b146-3e2b9d5e057b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:48.431416 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff139757-94d0-4120-b146-3e2b9d5e057b None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:48.431695 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff139757-94d0-4120-b146-3e2b9d5e057b None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:48.431965 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff139757-94d0-4120-b146-3e2b9d5e057b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:48.434570 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887237fc-26ef-463c-aaab-84b689d93575 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:48.455634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff139757-94d0-4120-b146-3e2b9d5e057b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:48.463471 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-ff139757-94d0-4120-b146-3e2b9d5e057b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:48.463471 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:48.463471 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:48.463471 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:48.463471 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:48.463471 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:48.463471 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:48.463471 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:48.463471 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:48.463471 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:48.463471 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:48.463471 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:48.463471 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:48.463471 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:48.463471 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:48.463471 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:48.464467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:48.464467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:48.464467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:48.464467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:48.464467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:48.464467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:48.464467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:48.464467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:48.464467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:48.464467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:48.464467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:48.464467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:48.464467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:48.464467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:48.464467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:48.464467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:48.465296 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:48.465296 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:48.465296 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:48.465296 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:48.465296 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:48.465296 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:48.465296 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:48.465296 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:48.465296 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:48.465296 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:48.465296 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:48.465296 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:02:48.466536 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:48.466612 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 193/385] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:02:48.475676 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff139757-94d0-4120-b146-3e2b9d5e057b None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:48.497959 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08544cc7-da55-47dd-91fc-a3d48644753e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:48.498833 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08544cc7-da55-47dd-91fc-a3d48644753e None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:48.499306 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08544cc7-da55-47dd-91fc-a3d48644753e None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:48.499589 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08544cc7-da55-47dd-91fc-a3d48644753e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:48.514826 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:48.521386 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08544cc7-da55-47dd-91fc-a3d48644753e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:48.535301 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-08544cc7-da55-47dd-91fc-a3d48644753e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:02:48.535301 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:48.535301 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:48.535301 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:48.535301 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:48.535301 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:48.535301 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:48.535301 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:48.535301 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:48.535301 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:48.535301 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:48.535301 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:48.535301 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:48.535301 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:48.535301 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:48.535301 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:48.536267 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:48.536267 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:48.536267 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:48.536267 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:48.536267 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:48.536267 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:48.536267 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:48.536267 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:48.536267 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:48.536267 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:48.536267 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:48.536267 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:48.536267 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:48.536267 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:48.536267 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:48.536267 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:48.537519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:48.537519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:48.537519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:48.537519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:48.537519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:48.537519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:48.537519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:48.537519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:48.537519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:48.537519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:48.537519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:02:48.537519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:02:48.537519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 193/386] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 27 21:02:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:02:48.555285 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08544cc7-da55-47dd-91fc-a3d48644753e None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:48.661846 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff139757-94d0-4120-b146-3e2b9d5e057b None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:48.733443 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5400e079-b6f8-4808-af59-53dbe4faaea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:48.734123 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400e079-b6f8-4808-af59-53dbe4faaea2 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:48.734401 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400e079-b6f8-4808-af59-53dbe4faaea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:48.734673 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400e079-b6f8-4808-af59-53dbe4faaea2 None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:48.735430 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5400e079-b6f8-4808-af59-53dbe4faaea2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:48.735736 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5400e079-b6f8-4808-af59-53dbe4faaea2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:48.737140 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5400e079-b6f8-4808-af59-53dbe4faaea2 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:48.740329 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08544cc7-da55-47dd-91fc-a3d48644753e None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:48.755354 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 194/387] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 27 21:02:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:48.760670 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5400e079-b6f8-4808-af59-53dbe4faaea2 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:48.810223 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba4fb36-1f2f-411f-899c-ce4ba8c96ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:48.810945 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba4fb36-1f2f-411f-899c-ce4ba8c96ccc None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:48.811225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba4fb36-1f2f-411f-899c-ce4ba8c96ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:48.811492 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba4fb36-1f2f-411f-899c-ce4ba8c96ccc None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:48.812247 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba4fb36-1f2f-411f-899c-ce4ba8c96ccc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:48.812549 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba4fb36-1f2f-411f-899c-ce4ba8c96ccc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:48.813979 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba4fb36-1f2f-411f-899c-ce4ba8c96ccc None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:48.841446 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 194/388] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:02:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:48.856290 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba4fb36-1f2f-411f-899c-ce4ba8c96ccc None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:48.907867 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:48.931077 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5400e079-b6f8-4808-af59-53dbe4faaea2 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:48.937985 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:48.971571 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ea0b8e-3b93-4114-9d91-5d3ddc5aa972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:48.972464 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea0b8e-3b93-4114-9d91-5d3ddc5aa972 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:48.972767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea0b8e-3b93-4114-9d91-5d3ddc5aa972 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:48.973069 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea0b8e-3b93-4114-9d91-5d3ddc5aa972 None admin] PATH_INFO: `/v3/groups/11aca19a34c34bc28cf9d777aef5f6ab` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:48.974494 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ea0b8e-3b93-4114-9d91-5d3ddc5aa972 None admin] RBAC: Authorizing `identity:get_group(group_id=11aca19a34c34bc28cf9d777aef5f6ab)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:48.976005 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ea0b8e-3b93-4114-9d91-5d3ddc5aa972 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:48.978496 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 195/389] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Jun 27 21:02:48 2022] GET /identity/v3/groups/11aca19a34c34bc28cf9d777aef5f6ab => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:48.984585 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ea0b8e-3b93-4114-9d91-5d3ddc5aa972 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:49.041221 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba4fb36-1f2f-411f-899c-ce4ba8c96ccc None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:49.112694 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce262d-f361-4c82-a4a0-7b9ec3eb6ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:49.113645 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce262d-f361-4c82-a4a0-7b9ec3eb6ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:49.113923 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce262d-f361-4c82-a4a0-7b9ec3eb6ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:49.114193 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce262d-f361-4c82-a4a0-7b9ec3eb6ee4 None admin] PATH_INFO: `/v3/groups/11aca19a34c34bc28cf9d777aef5f6ab` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:49.119184 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ce262d-f361-4c82-a4a0-7b9ec3eb6ee4 None admin] RBAC: Authorizing `identity:get_group(group_id=11aca19a34c34bc28cf9d777aef5f6ab)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:49.124259 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ce262d-f361-4c82-a4a0-7b9ec3eb6ee4 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:49.126055 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 195/390] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Jun 27 21:02:48 2022] GET /identity/v3/groups/11aca19a34c34bc28cf9d777aef5f6ab => generated 226 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:49.132198 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce262d-f361-4c82-a4a0-7b9ec3eb6ee4 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:49.148856 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ea0b8e-3b93-4114-9d91-5d3ddc5aa972 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:49.187290 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4219f387-e955-4ad0-a2dd-ad280859c581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:49.188080 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4219f387-e955-4ad0-a2dd-ad280859c581 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:49.188391 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4219f387-e955-4ad0-a2dd-ad280859c581 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:49.188778 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4219f387-e955-4ad0-a2dd-ad280859c581 None admin] PATH_INFO: `/v3/projects/04d09da302f44fefbe5b4502010ec050` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:49.192551 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4219f387-e955-4ad0-a2dd-ad280859c581 None admin] RBAC: Authorizing `identity:get_project(project_id=04d09da302f44fefbe5b4502010ec050)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:49.194270 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4219f387-e955-4ad0-a2dd-ad280859c581 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:49.197846 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 196/391] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:48 2022] GET /identity/v3/projects/04d09da302f44fefbe5b4502010ec050 => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:49.206957 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4219f387-e955-4ad0-a2dd-ad280859c581 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:49.252547 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:49.357754 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4219f387-e955-4ad0-a2dd-ad280859c581 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:49.389967 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:49.394523 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce262d-f361-4c82-a4a0-7b9ec3eb6ee4 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:49.400815 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eecdb34-5dc0-43db-acb2-0d25c57a641e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:49.401899 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eecdb34-5dc0-43db-acb2-0d25c57a641e None admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:49.402183 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eecdb34-5dc0-43db-acb2-0d25c57a641e None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:49.402456 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eecdb34-5dc0-43db-acb2-0d25c57a641e None admin] PATH_INFO: `/v3/projects/04d09da302f44fefbe5b4502010ec050/groups/11aca19a34c34bc28cf9d777aef5f6ab/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:49.406413 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eecdb34-5dc0-43db-acb2-0d25c57a641e None admin] RBAC: Authorizing `identity:create_grant(project_id=04d09da302f44fefbe5b4502010ec050, group_id=11aca19a34c34bc28cf9d777aef5f6ab, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:49.407899 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eecdb34-5dc0-43db-acb2-0d25c57a641e None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:49.431386 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 197/392] 10.222.0.13 () {58 vars in 1442 bytes} [Mon Jun 27 21:02:49 2022] PUT /identity/v3/projects/04d09da302f44fefbe5b4502010ec050/groups/11aca19a34c34bc28cf9d777aef5f6ab/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:02:49.460063 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604f8cce-5b5f-4639-821d-025e075c88d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:49.460875 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f8cce-5b5f-4639-821d-025e075c88d9 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:49.461236 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f8cce-5b5f-4639-821d-025e075c88d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:49.461515 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f8cce-5b5f-4639-821d-025e075c88d9 None admin] PATH_INFO: `/v3/projects/04d09da302f44fefbe5b4502010ec050` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:49.462740 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604f8cce-5b5f-4639-821d-025e075c88d9 None admin] RBAC: Authorizing `identity:get_project(project_id=04d09da302f44fefbe5b4502010ec050)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:49.464601 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604f8cce-5b5f-4639-821d-025e075c88d9 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:49.467724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 196/393] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:49 2022] GET /identity/v3/projects/04d09da302f44fefbe5b4502010ec050 => generated 298 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:49.472855 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eecdb34-5dc0-43db-acb2-0d25c57a641e None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:49.593898 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:49.628645 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eecdb34-5dc0-43db-acb2-0d25c57a641e None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:49.686894 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f6d977-5408-4ad4-a3c1-07123d983655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:49.687821 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6d977-5408-4ad4-a3c1-07123d983655 None admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:49.688103 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6d977-5408-4ad4-a3c1-07123d983655 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:49.688380 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f6d977-5408-4ad4-a3c1-07123d983655 None admin] PATH_INFO: `/v3/projects/04d09da302f44fefbe5b4502010ec050/groups/11aca19a34c34bc28cf9d777aef5f6ab/roles/5834784bf54d4806b88e32e0fb8e18fe` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:49.690699 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f6d977-5408-4ad4-a3c1-07123d983655 None admin] RBAC: Authorizing `identity:create_grant(project_id=04d09da302f44fefbe5b4502010ec050, group_id=11aca19a34c34bc28cf9d777aef5f6ab, role_id=5834784bf54d4806b88e32e0fb8e18fe)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:49.692282 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f6d977-5408-4ad4-a3c1-07123d983655 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:49.716509 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 198/394] 10.222.0.13 () {58 vars in 1442 bytes} [Mon Jun 27 21:02:49 2022] PUT /identity/v3/projects/04d09da302f44fefbe5b4502010ec050/groups/11aca19a34c34bc28cf9d777aef5f6ab/roles/5834784bf54d4806b88e32e0fb8e18fe => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:02:49.731159 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27609284-6877-4590-b1bd-609dd19924ff None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:49.731521 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27609284-6877-4590-b1bd-609dd19924ff None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:49.731716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27609284-6877-4590-b1bd-609dd19924ff None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:49.733206 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 197/395] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:49.750599 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6b541-9300-46ab-9839-d7df3bc4c10f None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:49.750958 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6b541-9300-46ab-9839-d7df3bc4c10f None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:49.751164 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b6b541-9300-46ab-9839-d7df3bc4c10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:49.779372 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-64b6b541-9300-46ab-9839-d7df3bc4c10f None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:49.780661 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-64b6b541-9300-46ab-9839-d7df3bc4c10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:49.883931 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 199/396] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:49.921438 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27609284-6877-4590-b1bd-609dd19924ff None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:50.050398 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27609284-6877-4590-b1bd-609dd19924ff None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:50.072038 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b0f98-4be0-4699-80d3-a9a3b4a1803c None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:50.072349 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b0f98-4be0-4699-80d3-a9a3b4a1803c None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:50.072585 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b0f98-4be0-4699-80d3-a9a3b4a1803c None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:50.074571 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 200/397] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:50.080074 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a4bf9-e5e6-4d1b-ada4-56f4783ac3e3 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:50.080366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a4bf9-e5e6-4d1b-ada4-56f4783ac3e3 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:50.080636 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a4bf9-e5e6-4d1b-ada4-56f4783ac3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:50.089158 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c82086-8e08-46e8-9e30-cb252d0b1bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:50.090101 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c82086-8e08-46e8-9e30-cb252d0b1bde None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:50.090491 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c82086-8e08-46e8-9e30-cb252d0b1bde None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:50.090770 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c82086-8e08-46e8-9e30-cb252d0b1bde None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:50.102971 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c82086-8e08-46e8-9e30-cb252d0b1bde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:50.112255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-20c82086-8e08-46e8-9e30-cb252d0b1bde None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:02:50.112255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:50.112255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:50.112255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:50.112255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:50.112255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:50.112255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:50.112255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:50.112255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:50.112255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:50.112255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:50.112255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:50.112255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:50.112255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:50.112255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:50.112255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:50.114409 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:50.114409 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:50.114409 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:50.114409 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:50.114409 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:50.114409 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:50.114409 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:50.114409 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:50.114409 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:50.114409 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:50.114409 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:50.114409 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:50.114409 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:50.114409 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:50.114409 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:50.114409 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:50.116284 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:50.116284 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:50.116284 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:50.116284 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:50.116284 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:50.116284 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:50.116284 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:50.116284 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:50.116284 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:50.116284 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:50.116284 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:02:50.116284 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:02:50.116284 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 198/398] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 27 21:02:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:50.120952 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-2e3a4bf9-e5e6-4d1b-ada4-56f4783ac3e3 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:50.121191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c82086-8e08-46e8-9e30-cb252d0b1bde None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:50.121439 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-2e3a4bf9-e5e6-4d1b-ada4-56f4783ac3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:50.152801 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:50.234328 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c82086-8e08-46e8-9e30-cb252d0b1bde None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:50.247008 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:50.262083 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 201/399] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:02:50.270790 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed3c02-3c7d-41e0-be3b-f1af9fa18767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:50.271444 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed3c02-3c7d-41e0-be3b-f1af9fa18767 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:50.271731 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed3c02-3c7d-41e0-be3b-f1af9fa18767 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:50.272003 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed3c02-3c7d-41e0-be3b-f1af9fa18767 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:50.272762 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ed3c02-3c7d-41e0-be3b-f1af9fa18767 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:50.273097 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ed3c02-3c7d-41e0-be3b-f1af9fa18767 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:50.274563 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ed3c02-3c7d-41e0-be3b-f1af9fa18767 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:50.286991 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 199/400] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:02:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:50.289416 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed3c02-3c7d-41e0-be3b-f1af9fa18767 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:50.289607 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a4bf9-e5e6-4d1b-ada4-56f4783ac3e3 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:50.389893 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:50.401178 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed3c02-3c7d-41e0-be3b-f1af9fa18767 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:50.435877 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a4bf9-e5e6-4d1b-ada4-56f4783ac3e3 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:50.439371 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d410aa-a4eb-4533-acdb-8f4ef6223986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:50.439976 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d410aa-a4eb-4533-acdb-8f4ef6223986 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:50.440252 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d410aa-a4eb-4533-acdb-8f4ef6223986 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:50.440523 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d410aa-a4eb-4533-acdb-8f4ef6223986 None admin] PATH_INFO: `/v3/groups/11aca19a34c34bc28cf9d777aef5f6ab` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:50.441771 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d410aa-a4eb-4533-acdb-8f4ef6223986 None admin] RBAC: Authorizing `identity:get_group(group_id=11aca19a34c34bc28cf9d777aef5f6ab)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:50.443256 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d410aa-a4eb-4533-acdb-8f4ef6223986 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:50.444970 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 200/401] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Jun 27 21:02:50 2022] GET /identity/v3/groups/11aca19a34c34bc28cf9d777aef5f6ab => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:50.451260 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76740db4-ffad-4609-8e90-ceb0371f40c3 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:50.451548 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76740db4-ffad-4609-8e90-ceb0371f40c3 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:50.451826 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76740db4-ffad-4609-8e90-ceb0371f40c3 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:50.454174 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 201/402] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:50 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:50.458604 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76740db4-ffad-4609-8e90-ceb0371f40c3 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:50.491172 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456afd25-a7e8-4ea0-a753-f39712d27545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:50.493063 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456afd25-a7e8-4ea0-a753-f39712d27545 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:50.493492 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456afd25-a7e8-4ea0-a753-f39712d27545 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:50.493911 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456afd25-a7e8-4ea0-a753-f39712d27545 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:50.508265 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456afd25-a7e8-4ea0-a753-f39712d27545 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:50.519662 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-456afd25-a7e8-4ea0-a753-f39712d27545 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:50.519662 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:50.519662 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:50.519662 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:50.519662 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:50.519662 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:50.519662 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:50.519662 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:50.519662 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:50.519662 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:50.519662 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:50.519662 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:50.519662 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:50.519662 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:50.519662 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:50.519662 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:50.520993 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:50.520993 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:50.520993 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:50.520993 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:50.520993 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:50.520993 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:50.520993 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:50.520993 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:50.520993 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:50.520993 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:50.520993 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:50.520993 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:50.520993 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:50.520993 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:50.520993 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:50.520993 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:50.522034 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:50.522034 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:50.522034 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:50.522034 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:50.522034 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:50.522034 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:50.522034 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:50.522034 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:50.522034 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:50.522034 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:50.522034 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:50.522034 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:02:50.525513 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 202/403] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:50.527093 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c083ed1-a786-474a-bdea-345b4c936619 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:50.527556 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c083ed1-a786-474a-bdea-345b4c936619 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:50.527995 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c083ed1-a786-474a-bdea-345b4c936619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:50.552080 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-5c083ed1-a786-474a-bdea-345b4c936619 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:50.552866 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-5c083ed1-a786-474a-bdea-345b4c936619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:50.595682 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76740db4-ffad-4609-8e90-ceb0371f40c3 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:50.640949 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 203/404] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:50.640949 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c083ed1-a786-474a-bdea-345b4c936619 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:50.641407 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:50.652557 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaa30cd-5c82-4720-9ee5-7f483de6c57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:50.653872 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa30cd-5c82-4720-9ee5-7f483de6c57d None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:50.654663 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa30cd-5c82-4720-9ee5-7f483de6c57d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:50.655115 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa30cd-5c82-4720-9ee5-7f483de6c57d None admin] PATH_INFO: `/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:50.661249 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaa30cd-5c82-4720-9ee5-7f483de6c57d None admin] RBAC: Authorizing `identity:get_project(project_id=8fc48df8fb1f4de59d32fefd09a8ecae)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:50.664064 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaa30cd-5c82-4720-9ee5-7f483de6c57d None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:50.671321 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 202/405] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:50 2022] GET /identity/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:50.672845 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaa30cd-5c82-4720-9ee5-7f483de6c57d None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:50.706761 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:50.785001 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c083ed1-a786-474a-bdea-345b4c936619 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:50.809248 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaa30cd-5c82-4720-9ee5-7f483de6c57d None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:50.835373 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d15d314-4a77-498f-aae6-392283439636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:50.836051 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15d314-4a77-498f-aae6-392283439636 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:50.836332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15d314-4a77-498f-aae6-392283439636 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:50.836596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15d314-4a77-498f-aae6-392283439636 None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:50.837396 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d15d314-4a77-498f-aae6-392283439636 None admin] RBAC: Adding query filter params (name=member) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:50.837695 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d15d314-4a77-498f-aae6-392283439636 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:50.839537 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d15d314-4a77-498f-aae6-392283439636 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:50.852970 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe762f2-b2bb-460a-9741-4c3af37d7e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:50.854204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe762f2-b2bb-460a-9741-4c3af37d7e92 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:50.854733 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe762f2-b2bb-460a-9741-4c3af37d7e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:50.855241 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe762f2-b2bb-460a-9741-4c3af37d7e92 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:50.856208 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 204/406] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 27 21:02:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:50.857172 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d15d314-4a77-498f-aae6-392283439636 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:50.870697 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe762f2-b2bb-460a-9741-4c3af37d7e92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:50.880945 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-3fe762f2-b2bb-460a-9741-4c3af37d7e92 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:02:50.880945 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:50.880945 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:50.880945 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:50.880945 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:50.880945 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:50.880945 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:50.880945 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:50.880945 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:50.880945 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:50.880945 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:50.880945 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:50.880945 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:50.880945 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:50.880945 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:50.880945 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:50.881853 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:50.881853 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:50.881853 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:50.881853 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:50.881853 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:50.881853 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:50.881853 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:50.881853 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:50.881853 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:50.881853 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:50.881853 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:50.881853 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:50.881853 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:50.881853 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:50.881853 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:50.881853 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:50.882655 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:50.882655 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:50.882655 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:50.882655 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:50.882655 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:50.882655 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:50.882655 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:50.882655 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:50.882655 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:50.882655 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:50.882655 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:02:50.882655 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:02:50.890976 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 203/407] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 27 21:02:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:50.890976 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe762f2-b2bb-460a-9741-4c3af37d7e92 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:50.929376 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:51.005217 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d15d314-4a77-498f-aae6-392283439636 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:51.007019 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:51.029536 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe762f2-b2bb-460a-9741-4c3af37d7e92 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:51.054282 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c7676-dfe5-4316-8cc0-a4208c2605d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:51.055105 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c7676-dfe5-4316-8cc0-a4208c2605d4 None admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:51.055664 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c7676-dfe5-4316-8cc0-a4208c2605d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:51.055946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c7676-dfe5-4316-8cc0-a4208c2605d4 None admin] PATH_INFO: `/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae/groups/11aca19a34c34bc28cf9d777aef5f6ab/roles/5834784bf54d4806b88e32e0fb8e18fe` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:51.059227 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964c7676-dfe5-4316-8cc0-a4208c2605d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=8fc48df8fb1f4de59d32fefd09a8ecae, group_id=11aca19a34c34bc28cf9d777aef5f6ab, role_id=5834784bf54d4806b88e32e0fb8e18fe)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:51.061283 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964c7676-dfe5-4316-8cc0-a4208c2605d4 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:51.070699 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac5ad11-c79a-459d-9afd-b85aa411ca55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:51.071866 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5ad11-c79a-459d-9afd-b85aa411ca55 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:51.072272 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5ad11-c79a-459d-9afd-b85aa411ca55 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:51.072643 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5ad11-c79a-459d-9afd-b85aa411ca55 None admin] PATH_INFO: `/v3/groups/11aca19a34c34bc28cf9d777aef5f6ab` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:51.074149 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac5ad11-c79a-459d-9afd-b85aa411ca55 None admin] RBAC: Authorizing `identity:get_group(group_id=11aca19a34c34bc28cf9d777aef5f6ab)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:51.075839 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac5ad11-c79a-459d-9afd-b85aa411ca55 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:51.082396 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 204/409] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Jun 27 21:02:50 2022] GET /identity/v3/groups/11aca19a34c34bc28cf9d777aef5f6ab => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:51.082608 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 205/409] 10.222.0.13 () {58 vars in 1442 bytes} [Mon Jun 27 21:02:50 2022] PUT /identity/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae/groups/11aca19a34c34bc28cf9d777aef5f6ab/roles/5834784bf54d4806b88e32e0fb8e18fe => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:02:51.083628 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c7676-dfe5-4316-8cc0-a4208c2605d4 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:51.083735 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac5ad11-c79a-459d-9afd-b85aa411ca55 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:51.203585 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac5ad11-c79a-459d-9afd-b85aa411ca55 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:51.241114 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f6c94-3b31-4920-984a-0cae75e08f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:51.241789 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f6c94-3b31-4920-984a-0cae75e08f87 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:51.242242 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f6c94-3b31-4920-984a-0cae75e08f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:51.242667 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f6c94-3b31-4920-984a-0cae75e08f87 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:51.243589 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f6c94-3b31-4920-984a-0cae75e08f87 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:51.244061 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f6c94-3b31-4920-984a-0cae75e08f87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:51.245726 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f6c94-3b31-4920-984a-0cae75e08f87 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:51.256390 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 205/410] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:51.263042 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f6c94-3b31-4920-984a-0cae75e08f87 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:51.286152 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:51.299344 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c7676-dfe5-4316-8cc0-a4208c2605d4 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:51.321402 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:51.380681 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f6c94-3b31-4920-984a-0cae75e08f87 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:51.391911 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70591d7c-ddbc-4be0-8c55-29bb34ccbb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:51.392814 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70591d7c-ddbc-4be0-8c55-29bb34ccbb64 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:51.393166 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70591d7c-ddbc-4be0-8c55-29bb34ccbb64 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:51.393821 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70591d7c-ddbc-4be0-8c55-29bb34ccbb64 None admin] PATH_INFO: `/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:51.395379 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70591d7c-ddbc-4be0-8c55-29bb34ccbb64 None admin] RBAC: Authorizing `identity:get_project(project_id=8fc48df8fb1f4de59d32fefd09a8ecae)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:51.397183 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70591d7c-ddbc-4be0-8c55-29bb34ccbb64 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:51.403839 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 206/411] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:51 2022] GET /identity/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae => generated 302 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:51.405066 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70591d7c-ddbc-4be0-8c55-29bb34ccbb64 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:51.428695 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d6f43-a1a4-4d0c-b6a8-c2cd3913dee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:51.430273 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d6f43-a1a4-4d0c-b6a8-c2cd3913dee2 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:51.431073 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d6f43-a1a4-4d0c-b6a8-c2cd3913dee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:51.431550 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d6f43-a1a4-4d0c-b6a8-c2cd3913dee2 None admin] PATH_INFO: `/v3/groups/2c24848f4d824e949955fab72678075d` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:51.433099 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173d6f43-a1a4-4d0c-b6a8-c2cd3913dee2 None admin] RBAC: Authorizing `identity:get_group(group_id=2c24848f4d824e949955fab72678075d)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:51.434872 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173d6f43-a1a4-4d0c-b6a8-c2cd3913dee2 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:51.437141 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 206/412] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Jun 27 21:02:51 2022] GET /identity/v3/groups/2c24848f4d824e949955fab72678075d => generated 229 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:51.445954 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d6f43-a1a4-4d0c-b6a8-c2cd3913dee2 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:51.571635 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70591d7c-ddbc-4be0-8c55-29bb34ccbb64 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:51.603031 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:51.622225 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb3e41-c81f-4d85-a979-6b230e00a0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:51.623131 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb3e41-c81f-4d85-a979-6b230e00a0c4 None admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:51.623416 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb3e41-c81f-4d85-a979-6b230e00a0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:51.623691 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb3e41-c81f-4d85-a979-6b230e00a0c4 None admin] PATH_INFO: `/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae/groups/11aca19a34c34bc28cf9d777aef5f6ab/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:51.630891 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb3e41-c81f-4d85-a979-6b230e00a0c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=8fc48df8fb1f4de59d32fefd09a8ecae, group_id=11aca19a34c34bc28cf9d777aef5f6ab, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:51.632357 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb3e41-c81f-4d85-a979-6b230e00a0c4 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:51.642596 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d6f43-a1a4-4d0c-b6a8-c2cd3913dee2 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:51.656382 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 207/413] 10.222.0.13 () {58 vars in 1442 bytes} [Mon Jun 27 21:02:51 2022] PUT /identity/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae/groups/11aca19a34c34bc28cf9d777aef5f6ab/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:02:51.707822 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1712a722-30f5-4e6b-81a1-74047db0b3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:51.708708 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1712a722-30f5-4e6b-81a1-74047db0b3de None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:51.709008 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1712a722-30f5-4e6b-81a1-74047db0b3de None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:51.709335 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1712a722-30f5-4e6b-81a1-74047db0b3de None admin] PATH_INFO: `/v3/projects/b6e5cf8b46f34e30a9b900f3cd1507c0` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:51.712677 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1712a722-30f5-4e6b-81a1-74047db0b3de None admin] RBAC: Authorizing `identity:get_project(project_id=b6e5cf8b46f34e30a9b900f3cd1507c0)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:51.714307 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1712a722-30f5-4e6b-81a1-74047db0b3de None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:51.716434 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 207/414] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:51 2022] GET /identity/v3/projects/b6e5cf8b46f34e30a9b900f3cd1507c0 => generated 344 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:51.722406 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb3e41-c81f-4d85-a979-6b230e00a0c4 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:51.803080 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:51.887789 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb3e41-c81f-4d85-a979-6b230e00a0c4 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:51.908497 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a855b26-c609-4847-97c8-d105f13d64c0 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:51.908837 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a855b26-c609-4847-97c8-d105f13d64c0 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:51.909096 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a855b26-c609-4847-97c8-d105f13d64c0 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:51.910730 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 208/415] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:51.916190 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb4f23-b403-4af2-aa70-0cf922892a46 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:51.916469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb4f23-b403-4af2-aa70-0cf922892a46 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:51.916750 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb4f23-b403-4af2-aa70-0cf922892a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:51.944320 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b9dfe-775b-43ca-9896-53bf3bec7922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:51.945287 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b9dfe-775b-43ca-9896-53bf3bec7922 None admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:51.945570 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b9dfe-775b-43ca-9896-53bf3bec7922 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:51.945841 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b9dfe-775b-43ca-9896-53bf3bec7922 None admin] PATH_INFO: `/v3/projects/b6e5cf8b46f34e30a9b900f3cd1507c0/groups/2c24848f4d824e949955fab72678075d/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:51.951533 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568b9dfe-775b-43ca-9896-53bf3bec7922 None admin] RBAC: Authorizing `identity:create_grant(project_id=b6e5cf8b46f34e30a9b900f3cd1507c0, group_id=2c24848f4d824e949955fab72678075d, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:51.953073 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568b9dfe-775b-43ca-9896-53bf3bec7922 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:51.956774 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-27eb4f23-b403-4af2-aa70-0cf922892a46 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:51.958052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-27eb4f23-b403-4af2-aa70-0cf922892a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:51.976300 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 208/416] 10.222.0.13 () {58 vars in 1442 bytes} [Mon Jun 27 21:02:51 2022] PUT /identity/v3/projects/b6e5cf8b46f34e30a9b900f3cd1507c0/groups/2c24848f4d824e949955fab72678075d/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:02:52.060508 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 209/417] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:02:52.083610 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b9dfe-775b-43ca-9896-53bf3bec7922 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:52.185176 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf91e9b-0e9f-4935-8562-854b614a79ab None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:52.185726 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf91e9b-0e9f-4935-8562-854b614a79ab None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:52.185726 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf91e9b-0e9f-4935-8562-854b614a79ab None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:52.187103 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 210/418] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:52.195336 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e44776-bdf5-4ba7-8bd9-3911cb9c6d42 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:52.195621 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e44776-bdf5-4ba7-8bd9-3911cb9c6d42 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:52.195896 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e44776-bdf5-4ba7-8bd9-3911cb9c6d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:52.218107 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-44e44776-bdf5-4ba7-8bd9-3911cb9c6d42 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:52.219327 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-44e44776-bdf5-4ba7-8bd9-3911cb9c6d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:52.222321 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:52.226387 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b9dfe-775b-43ca-9896-53bf3bec7922 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:52.266330 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d7cf30-3aef-4092-b8db-fd206d7d456e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:52.267149 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7cf30-3aef-4092-b8db-fd206d7d456e None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:52.267427 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7cf30-3aef-4092-b8db-fd206d7d456e None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:52.267699 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7cf30-3aef-4092-b8db-fd206d7d456e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:52.287398 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d7cf30-3aef-4092-b8db-fd206d7d456e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:52.299241 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-88d7cf30-3aef-4092-b8db-fd206d7d456e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:52.299241 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:52.299241 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:52.299241 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:52.299241 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:52.299241 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:52.299241 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:52.299241 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:52.299241 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:52.299241 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:52.299241 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:52.299241 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:52.299241 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:52.299241 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:52.299241 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:52.299241 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:52.300721 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:52.300721 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:52.300721 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:52.300721 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:52.300721 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:52.300721 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:52.300721 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:52.300721 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:52.300721 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:52.300721 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:52.300721 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:52.300721 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:52.300721 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:52.300721 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:52.300721 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:52.300721 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:52.301556 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:52.301556 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:52.301556 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:52.301556 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:52.301556 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:52.301556 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:52.301556 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:52.301556 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:52.301556 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:52.301556 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:52.301556 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:52.301556 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:02:52.304976 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 209/419] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:52.309381 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d7cf30-3aef-4092-b8db-fd206d7d456e None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:52.363338 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 211/420] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:02:52.390245 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e44776-bdf5-4ba7-8bd9-3911cb9c6d42 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:52.433491 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:52.530599 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d7cf30-3aef-4092-b8db-fd206d7d456e None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:52.576297 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611fef84-ba0a-4521-a718-ae2501acb9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:52.576998 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fef84-ba0a-4521-a718-ae2501acb9c3 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:52.577282 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fef84-ba0a-4521-a718-ae2501acb9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:52.577556 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fef84-ba0a-4521-a718-ae2501acb9c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:52.578320 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611fef84-ba0a-4521-a718-ae2501acb9c3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:52.578630 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611fef84-ba0a-4521-a718-ae2501acb9c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:52.580030 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611fef84-ba0a-4521-a718-ae2501acb9c3 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:52.582995 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e44776-bdf5-4ba7-8bd9-3911cb9c6d42 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:52.600022 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 210/421] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 27 21:02:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:52.606217 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611fef84-ba0a-4521-a718-ae2501acb9c3 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:52.635091 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:52.669649 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5f0ac1-56fa-414e-934e-2487d3953bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:52.670878 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f0ac1-56fa-414e-934e-2487d3953bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:52.671359 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f0ac1-56fa-414e-934e-2487d3953bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:52.671828 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f0ac1-56fa-414e-934e-2487d3953bf0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:52.695707 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5f0ac1-56fa-414e-934e-2487d3953bf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:52.713810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-5c5f0ac1-56fa-414e-934e-2487d3953bf0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:02:52.713810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:52.713810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:52.713810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:52.713810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:52.713810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:52.713810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:52.713810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:52.713810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:52.713810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:52.713810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:52.713810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:52.713810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:52.713810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:52.713810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:52.713810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:52.714802 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:52.714802 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:52.714802 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:52.714802 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:52.714802 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:52.714802 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:52.714802 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:52.714802 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:52.714802 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:52.714802 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:52.714802 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:52.714802 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:52.714802 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:52.714802 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:52.714802 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:52.714802 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:52.715604 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:52.715604 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:52.715604 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:52.715604 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:52.715604 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:52.715604 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:52.715604 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:52.715604 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:52.715604 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:52.715604 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:52.715604 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:02:52.715604 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:02:52.718495 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 212/422] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 27 21:02:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:52.720686 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611fef84-ba0a-4521-a718-ae2501acb9c3 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:52.726494 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5f0ac1-56fa-414e-934e-2487d3953bf0 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:52.766169 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c7af6f-df79-45ef-8485-f029e9779adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:52.766838 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7af6f-df79-45ef-8485-f029e9779adf None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:52.767122 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7af6f-df79-45ef-8485-f029e9779adf None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:52.767391 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c7af6f-df79-45ef-8485-f029e9779adf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:52.860033 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:52.906158 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c7af6f-df79-45ef-8485-f029e9779adf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:52.908882 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c7af6f-df79-45ef-8485-f029e9779adf None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:52.923524 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5f0ac1-56fa-414e-934e-2487d3953bf0 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:52.952008 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:52.993717 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3a7840-4f57-4146-989d-a250ff27a09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:52.994823 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a7840-4f57-4146-989d-a250ff27a09c None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:52.995310 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a7840-4f57-4146-989d-a250ff27a09c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:52.995814 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a7840-4f57-4146-989d-a250ff27a09c None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:52.996969 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3a7840-4f57-4146-989d-a250ff27a09c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:52.997478 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3a7840-4f57-4146-989d-a250ff27a09c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:52.999177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3a7840-4f57-4146-989d-a250ff27a09c None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:53.008516 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 213/423] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:02:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:53.014809 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3a7840-4f57-4146-989d-a250ff27a09c None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:53.108722 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 211/424] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 21:02:53.114196 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c7af6f-df79-45ef-8485-f029e9779adf None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:53.131629 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3a7840-4f57-4146-989d-a250ff27a09c None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:53.183898 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2f4719-a447-4de0-ad00-acd6c95a16c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:53.185096 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f4719-a447-4de0-ad00-acd6c95a16c5 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:53.185581 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f4719-a447-4de0-ad00-acd6c95a16c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:53.186549 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f4719-a447-4de0-ad00-acd6c95a16c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:53.191130 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:53.239178 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c7af6f-df79-45ef-8485-f029e9779adf None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:53.262489 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:53.277397 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404df14-fd5f-4706-909f-78d0f1fad51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:53.278273 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404df14-fd5f-4706-909f-78d0f1fad51d None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:53.278748 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404df14-fd5f-4706-909f-78d0f1fad51d None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:53.279190 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404df14-fd5f-4706-909f-78d0f1fad51d None admin] PATH_INFO: `/v3/projects/04d09da302f44fefbe5b4502010ec050` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:53.280604 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6404df14-fd5f-4706-909f-78d0f1fad51d None admin] RBAC: Authorizing `identity:get_project(project_id=04d09da302f44fefbe5b4502010ec050)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:53.282347 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6404df14-fd5f-4706-909f-78d0f1fad51d None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:53.284740 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 212/425] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:53 2022] GET /identity/v3/projects/04d09da302f44fefbe5b4502010ec050 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:53.290456 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404df14-fd5f-4706-909f-78d0f1fad51d None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:53.327028 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2f4719-a447-4de0-ad00-acd6c95a16c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:53.328767 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2f4719-a447-4de0-ad00-acd6c95a16c5 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:53.396893 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404df14-fd5f-4706-909f-78d0f1fad51d None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:53.435983 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca2506-ef65-4d8b-86ef-160cd73406b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:53.437239 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca2506-ef65-4d8b-86ef-160cd73406b1 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:53.437662 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca2506-ef65-4d8b-86ef-160cd73406b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:53.438086 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca2506-ef65-4d8b-86ef-160cd73406b1 None admin] PATH_INFO: `/v3/groups/11aca19a34c34bc28cf9d777aef5f6ab` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:53.442338 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ca2506-ef65-4d8b-86ef-160cd73406b1 None admin] RBAC: Authorizing `identity:get_group(group_id=11aca19a34c34bc28cf9d777aef5f6ab)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:53.443936 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ca2506-ef65-4d8b-86ef-160cd73406b1 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:53.446821 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 213/426] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Jun 27 21:02:53 2022] GET /identity/v3/groups/11aca19a34c34bc28cf9d777aef5f6ab => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:53.454647 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca2506-ef65-4d8b-86ef-160cd73406b1 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:53.539088 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 214/427] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:02:53.549860 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2f4719-a447-4de0-ad00-acd6c95a16c5 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:53.583859 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:53.626119 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:53.654277 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca2506-ef65-4d8b-86ef-160cd73406b1 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:53.697251 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b368d6b8-99dd-41f7-843e-62fc2ecebf50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:53.699070 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368d6b8-99dd-41f7-843e-62fc2ecebf50 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:53.699527 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368d6b8-99dd-41f7-843e-62fc2ecebf50 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:53.700070 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368d6b8-99dd-41f7-843e-62fc2ecebf50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:53.701238 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b368d6b8-99dd-41f7-843e-62fc2ecebf50 None admin] RBAC: Adding query filter params (group.id=11aca19a34c34bc28cf9d777aef5f6ab, role.id=e4414bde4df44de8aacb36c585a93e07, scope.project.id=04d09da302f44fefbe5b4502010ec050) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:53.701697 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b368d6b8-99dd-41f7-843e-62fc2ecebf50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:53.703559 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b368d6b8-99dd-41f7-843e-62fc2ecebf50 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:53.715874 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 214/428] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Jun 27 21:02:53 2022] GET /identity/v3/role_assignments?group.id=11aca19a34c34bc28cf9d777aef5f6ab&scope.project.id=04d09da302f44fefbe5b4502010ec050&role.id=e4414bde4df44de8aacb36c585a93e07 => generated 611 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:53.752879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2f4719-a447-4de0-ad00-acd6c95a16c5 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:53.772318 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380fd458-435b-45a7-9e62-8dabe7fb2bfd None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:53.772646 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380fd458-435b-45a7-9e62-8dabe7fb2bfd None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:53.772878 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380fd458-435b-45a7-9e62-8dabe7fb2bfd None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:53.774457 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 215/429] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:02:53.780446 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b38f8-251e-4441-9f18-6af0ace87d0d None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:53.780749 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b38f8-251e-4441-9f18-6af0ace87d0d None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:53.781025 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b38f8-251e-4441-9f18-6af0ace87d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:53.814217 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-c21b38f8-251e-4441-9f18-6af0ace87d0d None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:53.815847 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-c21b38f8-251e-4441-9f18-6af0ace87d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:53.828103 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882fcc1d-7f3a-4771-9a23-5ed5145fd6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:53.829489 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882fcc1d-7f3a-4771-9a23-5ed5145fd6d0 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:53.830018 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882fcc1d-7f3a-4771-9a23-5ed5145fd6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:53.830514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882fcc1d-7f3a-4771-9a23-5ed5145fd6d0 None admin] PATH_INFO: `/v3/projects/04d09da302f44fefbe5b4502010ec050` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:53.832114 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882fcc1d-7f3a-4771-9a23-5ed5145fd6d0 None admin] RBAC: Authorizing `identity:get_project(project_id=04d09da302f44fefbe5b4502010ec050)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:53.836503 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882fcc1d-7f3a-4771-9a23-5ed5145fd6d0 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:53.839253 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 215/430] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:53 2022] GET /identity/v3/projects/04d09da302f44fefbe5b4502010ec050 => generated 298 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:53.845769 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882fcc1d-7f3a-4771-9a23-5ed5145fd6d0 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:53.958146 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 216/431] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:02:53.984572 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21b38f8-251e-4441-9f18-6af0ace87d0d None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:54.013889 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882fcc1d-7f3a-4771-9a23-5ed5145fd6d0 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:54.015518 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:54.051943 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:54.066806 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5044fa89-a538-42a7-9c24-5fc25803ba71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:54.067463 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5044fa89-a538-42a7-9c24-5fc25803ba71 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:54.067743 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5044fa89-a538-42a7-9c24-5fc25803ba71 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:54.068016 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5044fa89-a538-42a7-9c24-5fc25803ba71 None admin] PATH_INFO: `/v3/groups/11aca19a34c34bc28cf9d777aef5f6ab` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:54.069372 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5044fa89-a538-42a7-9c24-5fc25803ba71 None admin] RBAC: Authorizing `identity:get_group(group_id=11aca19a34c34bc28cf9d777aef5f6ab)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:54.070844 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5044fa89-a538-42a7-9c24-5fc25803ba71 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:54.072506 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 216/432] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Jun 27 21:02:53 2022] GET /identity/v3/groups/11aca19a34c34bc28cf9d777aef5f6ab => generated 226 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:54.078848 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5044fa89-a538-42a7-9c24-5fc25803ba71 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:54.126323 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21b38f8-251e-4441-9f18-6af0ace87d0d None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:54.183816 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91721b99-27fc-436f-9d2c-0950ea327353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:54.185355 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91721b99-27fc-436f-9d2c-0950ea327353 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:54.185857 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91721b99-27fc-436f-9d2c-0950ea327353 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:54.186556 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91721b99-27fc-436f-9d2c-0950ea327353 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:54.205191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5044fa89-a538-42a7-9c24-5fc25803ba71 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:54.206017 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91721b99-27fc-436f-9d2c-0950ea327353 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:54.217061 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-91721b99-27fc-436f-9d2c-0950ea327353 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:02:54.217061 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:54.217061 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:54.217061 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:54.217061 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:54.217061 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:54.217061 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:54.217061 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:54.217061 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:54.217061 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:54.217061 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:54.217061 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:54.217061 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:54.217061 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:54.217061 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:54.217061 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:54.218365 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:54.218365 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:54.218365 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:54.218365 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:54.218365 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:54.218365 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:54.218365 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:54.218365 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:54.218365 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:54.218365 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:54.218365 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:54.218365 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:54.218365 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:54.218365 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:54.218365 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:54.218365 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:54.219239 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:54.219239 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:54.219239 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:54.219239 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:54.219239 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:54.219239 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:54.219239 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:54.219239 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:54.219239 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:54.219239 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:54.219239 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:02:54.219239 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:02:54.219239 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 217/433] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 27 21:02:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:54.226103 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91721b99-27fc-436f-9d2c-0950ea327353 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:54.267397 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ce1aac-1581-4ef9-8e7f-f01e876e0162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:54.267397 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce1aac-1581-4ef9-8e7f-f01e876e0162 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:54.267397 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce1aac-1581-4ef9-8e7f-f01e876e0162 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:54.267722 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce1aac-1581-4ef9-8e7f-f01e876e0162 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:54.268566 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ce1aac-1581-4ef9-8e7f-f01e876e0162 None admin] RBAC: Adding query filter params (group.id=11aca19a34c34bc28cf9d777aef5f6ab, role.id=5834784bf54d4806b88e32e0fb8e18fe, scope.project.id=04d09da302f44fefbe5b4502010ec050) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:54.273185 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ce1aac-1581-4ef9-8e7f-f01e876e0162 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:54.273185 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ce1aac-1581-4ef9-8e7f-f01e876e0162 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:54.283169 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 217/434] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Jun 27 21:02:54 2022] GET /identity/v3/role_assignments?group.id=11aca19a34c34bc28cf9d777aef5f6ab&scope.project.id=04d09da302f44fefbe5b4502010ec050&role.id=5834784bf54d4806b88e32e0fb8e18fe => generated 611 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:54.316671 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9eeb64-3b31-4154-9be2-30f058c649ef None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:54.317383 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9eeb64-3b31-4154-9be2-30f058c649ef None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:54.317860 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9eeb64-3b31-4154-9be2-30f058c649ef None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:54.319973 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 218/435] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:02:54.327169 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dccc986-cacb-4334-9eaf-3d21e0ac0ca2 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:54.327492 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dccc986-cacb-4334-9eaf-3d21e0ac0ca2 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:54.327732 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dccc986-cacb-4334-9eaf-3d21e0ac0ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:54.348316 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-3dccc986-cacb-4334-9eaf-3d21e0ac0ca2 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:54.349666 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-3dccc986-cacb-4334-9eaf-3d21e0ac0ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:54.364440 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91721b99-27fc-436f-9d2c-0950ea327353 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:54.381029 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:54.387783 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:54.414752 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9edb556-c486-4da4-a238-7da0ffed2168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:54.415544 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9edb556-c486-4da4-a238-7da0ffed2168 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:54.415695 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9edb556-c486-4da4-a238-7da0ffed2168 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:54.416072 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9edb556-c486-4da4-a238-7da0ffed2168 None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:54.416744 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9edb556-c486-4da4-a238-7da0ffed2168 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:54.417083 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9edb556-c486-4da4-a238-7da0ffed2168 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:54.418625 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9edb556-c486-4da4-a238-7da0ffed2168 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:54.428486 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 218/436] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:02:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:54.434240 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9edb556-c486-4da4-a238-7da0ffed2168 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:54.451762 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 219/437] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:54.476619 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dccc986-cacb-4334-9eaf-3d21e0ac0ca2 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:54.571240 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9edb556-c486-4da4-a238-7da0ffed2168 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:54.614560 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dccc986-cacb-4334-9eaf-3d21e0ac0ca2 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:54.616509 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6589b3e3-ae5e-4aaf-a3ca-1eda98fe5c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:54.618896 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6589b3e3-ae5e-4aaf-a3ca-1eda98fe5c40 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:54.618896 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6589b3e3-ae5e-4aaf-a3ca-1eda98fe5c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:54.618896 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6589b3e3-ae5e-4aaf-a3ca-1eda98fe5c40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:54.653948 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b5b792-419e-43c1-889e-c784972f126c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:54.654804 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5b792-419e-43c1-889e-c784972f126c None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:54.655094 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5b792-419e-43c1-889e-c784972f126c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:54.655418 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5b792-419e-43c1-889e-c784972f126c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:54.666181 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:54.671260 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b5b792-419e-43c1-889e-c784972f126c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:54.678597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-e9b5b792-419e-43c1-889e-c784972f126c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:02:54.678597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:54.678597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:54.678597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:54.678597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:54.678597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:54.678597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:54.678597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:54.678597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:54.678597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:54.678597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:54.678597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:54.678597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:54.678597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:54.678597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:54.678597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:54.679542 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:54.679542 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:54.679542 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:54.679542 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:54.679542 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:54.679542 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:54.679542 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:54.679542 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:54.679542 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:54.679542 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:54.679542 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:54.679542 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:54.679542 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:54.679542 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:54.679542 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:54.679542 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:54.680345 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:54.680345 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:54.680345 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:54.680345 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:54.680345 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:54.680345 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:54.680345 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:54.680345 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:54.680345 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:54.680345 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:54.680345 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:02:54.680345 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:02:54.680345 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 220/438] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 27 21:02:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:02:54.685367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf164c-cf48-439d-89da-bc94fdbcab09 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:54.685367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf164c-cf48-439d-89da-bc94fdbcab09 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:54.685367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf164c-cf48-439d-89da-bc94fdbcab09 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:54.685367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 221/439] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:54.687782 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf164c-cf48-439d-89da-bc94fdbcab09 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:54.698406 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6589b3e3-ae5e-4aaf-a3ca-1eda98fe5c40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:54.700886 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6589b3e3-ae5e-4aaf-a3ca-1eda98fe5c40 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:54.717256 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:54.809004 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf164c-cf48-439d-89da-bc94fdbcab09 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:54.818776 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 219/440] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:02:54.820998 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c3df6-d9f7-458e-808a-1b15896ce815 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:54.821289 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c3df6-d9f7-458e-808a-1b15896ce815 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:54.821563 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c3df6-d9f7-458e-808a-1b15896ce815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:54.837346 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-035c3df6-d9f7-458e-808a-1b15896ce815 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:54.838533 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-035c3df6-d9f7-458e-808a-1b15896ce815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:54.842479 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77cdea-bdd4-4866-b693-7c49b663b6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:54.843153 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77cdea-bdd4-4866-b693-7c49b663b6fc None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:54.843434 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77cdea-bdd4-4866-b693-7c49b663b6fc None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:54.843707 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77cdea-bdd4-4866-b693-7c49b663b6fc None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:54.844457 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd77cdea-bdd4-4866-b693-7c49b663b6fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:54.844763 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd77cdea-bdd4-4866-b693-7c49b663b6fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:54.846173 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd77cdea-bdd4-4866-b693-7c49b663b6fc None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:54.855369 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 222/441] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:54.856103 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77cdea-bdd4-4866-b693-7c49b663b6fc None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:54.909617 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 220/442] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:54.910457 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c3df6-d9f7-458e-808a-1b15896ce815 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:54.912689 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:54.940840 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:54.956593 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77cdea-bdd4-4866-b693-7c49b663b6fc None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:54.990093 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0518644-44b2-46af-8047-55233a705937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:54.991067 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0518644-44b2-46af-8047-55233a705937 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:54.991344 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0518644-44b2-46af-8047-55233a705937 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:54.991613 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0518644-44b2-46af-8047-55233a705937 None admin] PATH_INFO: `/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:54.992780 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0518644-44b2-46af-8047-55233a705937 None admin] RBAC: Authorizing `identity:get_project(project_id=8fc48df8fb1f4de59d32fefd09a8ecae)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:54.994248 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0518644-44b2-46af-8047-55233a705937 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:54.995904 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 223/443] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:54 2022] GET /identity/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:54.996711 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0518644-44b2-46af-8047-55233a705937 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:55.020521 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c3df6-d9f7-458e-808a-1b15896ce815 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:55.055654 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f712c-fb58-4fe4-932e-6ed9ea73a402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:55.056413 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f712c-fb58-4fe4-932e-6ed9ea73a402 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:55.056695 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f712c-fb58-4fe4-932e-6ed9ea73a402 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:55.057005 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f712c-fb58-4fe4-932e-6ed9ea73a402 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:55.100220 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0518644-44b2-46af-8047-55233a705937 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:55.122864 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f712c-fb58-4fe4-932e-6ed9ea73a402 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:55.124324 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f712c-fb58-4fe4-932e-6ed9ea73a402 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:55.126192 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:55.133525 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d7077a-d9c3-4750-83c7-f972d940c074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:55.134333 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7077a-d9c3-4750-83c7-f972d940c074 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:55.134617 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7077a-d9c3-4750-83c7-f972d940c074 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:55.134892 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7077a-d9c3-4750-83c7-f972d940c074 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:55.145272 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d7077a-d9c3-4750-83c7-f972d940c074 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:02:55.153101 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-64d7077a-d9c3-4750-83c7-f972d940c074 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:55.153101 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:55.153101 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:55.153101 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:55.153101 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:55.153101 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:55.153101 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:55.153101 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:55.153101 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:55.153101 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:55.153101 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:55.153101 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:55.153101 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:02:55.153101 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:02:55.153101 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:02:55.153101 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:02:55.154191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:02:55.154191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:02:55.154191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:02:55.154191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:02:55.154191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:55.154191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:55.154191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:55.154191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:55.154191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:55.154191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:55.154191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:55.154191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:55.154191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:55.154191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:55.154191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:55.154191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:55.155220 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:55.155220 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:55.155220 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:55.155220 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:55.155220 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:02:55.155220 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:02:55.155220 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:02:55.155220 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:02:55.155220 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:02:55.155220 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:02:55.155220 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 27 21:02:55.155220 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:02:55.159353 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 224/444] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:02:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:55.160192 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d7077a-d9c3-4750-83c7-f972d940c074 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:55.176112 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:55.220557 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 221/445] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:02:55.221425 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f712c-fb58-4fe4-932e-6ed9ea73a402 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:55.259342 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d7077a-d9c3-4750-83c7-f972d940c074 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:55.292870 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd99ad60-59d8-44b0-a24c-b2ed9095618d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:55.293765 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99ad60-59d8-44b0-a24c-b2ed9095618d None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:55.294039 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99ad60-59d8-44b0-a24c-b2ed9095618d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:55.294314 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99ad60-59d8-44b0-a24c-b2ed9095618d None admin] PATH_INFO: `/v3/groups/11aca19a34c34bc28cf9d777aef5f6ab` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:55.295505 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd99ad60-59d8-44b0-a24c-b2ed9095618d None admin] RBAC: Authorizing `identity:get_group(group_id=11aca19a34c34bc28cf9d777aef5f6ab)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:55.297016 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd99ad60-59d8-44b0-a24c-b2ed9095618d None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:55.298647 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 225/446] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Jun 27 21:02:55 2022] GET /identity/v3/groups/11aca19a34c34bc28cf9d777aef5f6ab => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:55.299405 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd99ad60-59d8-44b0-a24c-b2ed9095618d None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:55.327221 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f712c-fb58-4fe4-932e-6ed9ea73a402 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:55.328785 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:55.366431 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81261242-26bb-46ed-99d5-525353bbde4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:55.366846 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81261242-26bb-46ed-99d5-525353bbde4d None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:55.367119 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81261242-26bb-46ed-99d5-525353bbde4d None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:55.367368 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81261242-26bb-46ed-99d5-525353bbde4d None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:55.368111 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81261242-26bb-46ed-99d5-525353bbde4d None admin] RBAC: Adding query filter params (name=member) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:55.368414 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81261242-26bb-46ed-99d5-525353bbde4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:55.369812 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81261242-26bb-46ed-99d5-525353bbde4d None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:55.378692 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 222/447] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 27 21:02:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:55.379509 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81261242-26bb-46ed-99d5-525353bbde4d None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:55.396589 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:55.416620 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd99ad60-59d8-44b0-a24c-b2ed9095618d None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:55.452936 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21b325-6001-4da3-b901-586469a1f072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:55.454010 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21b325-6001-4da3-b901-586469a1f072 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:55.454345 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21b325-6001-4da3-b901-586469a1f072 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:55.454650 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21b325-6001-4da3-b901-586469a1f072 None admin] PATH_INFO: `/v3/projects/b6e5cf8b46f34e30a9b900f3cd1507c0` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:55.455868 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a21b325-6001-4da3-b901-586469a1f072 None admin] RBAC: Authorizing `identity:get_project(project_id=b6e5cf8b46f34e30a9b900f3cd1507c0)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:55.457362 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a21b325-6001-4da3-b901-586469a1f072 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:55.459077 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 226/448] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:55 2022] GET /identity/v3/projects/b6e5cf8b46f34e30a9b900f3cd1507c0 => generated 344 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:55.459845 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21b325-6001-4da3-b901-586469a1f072 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:55.488785 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81261242-26bb-46ed-99d5-525353bbde4d None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:55.527426 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1edfa-7568-45f5-b75d-b45f1e382d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:55.528121 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1edfa-7568-45f5-b75d-b45f1e382d6b None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:55.528516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1edfa-7568-45f5-b75d-b45f1e382d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:55.528893 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1edfa-7568-45f5-b75d-b45f1e382d6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:55.529751 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a1edfa-7568-45f5-b75d-b45f1e382d6b None admin] RBAC: Adding query filter params (group.id=11aca19a34c34bc28cf9d777aef5f6ab, role.id=5834784bf54d4806b88e32e0fb8e18fe, scope.project.id=8fc48df8fb1f4de59d32fefd09a8ecae) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:55.530093 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a1edfa-7568-45f5-b75d-b45f1e382d6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:55.531670 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a1edfa-7568-45f5-b75d-b45f1e382d6b None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:55.538344 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 223/449] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Jun 27 21:02:55 2022] GET /identity/v3/role_assignments?group.id=11aca19a34c34bc28cf9d777aef5f6ab&scope.project.id=8fc48df8fb1f4de59d32fefd09a8ecae&role.id=5834784bf54d4806b88e32e0fb8e18fe => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:55.539192 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1edfa-7568-45f5-b75d-b45f1e382d6b None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:55.571925 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21b325-6001-4da3-b901-586469a1f072 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:55.577530 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:55.623382 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5003e334-4ae4-42e2-a233-df6f10489c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:55.623967 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003e334-4ae4-42e2-a233-df6f10489c69 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:55.624250 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003e334-4ae4-42e2-a233-df6f10489c69 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:55.624525 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003e334-4ae4-42e2-a233-df6f10489c69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:55.647015 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:55.670872 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1edfa-7568-45f5-b75d-b45f1e382d6b None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:55.716646 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5003e334-4ae4-42e2-a233-df6f10489c69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:55.718792 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5003e334-4ae4-42e2-a233-df6f10489c69 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:55.719505 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45635de-bb44-4fc2-84d7-b97a38ad74a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:55.720433 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45635de-bb44-4fc2-84d7-b97a38ad74a6 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:55.720674 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45635de-bb44-4fc2-84d7-b97a38ad74a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:55.721839 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45635de-bb44-4fc2-84d7-b97a38ad74a6 None admin] PATH_INFO: `/v3/groups/2c24848f4d824e949955fab72678075d` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:55.722374 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45635de-bb44-4fc2-84d7-b97a38ad74a6 None admin] RBAC: Authorizing `identity:get_group(group_id=2c24848f4d824e949955fab72678075d)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:55.723885 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45635de-bb44-4fc2-84d7-b97a38ad74a6 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:55.725446 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 224/450] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Jun 27 21:02:55 2022] GET /identity/v3/groups/2c24848f4d824e949955fab72678075d => generated 229 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:55.731861 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45635de-bb44-4fc2-84d7-b97a38ad74a6 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:55.836599 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 227/451] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:02:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:02:55.839170 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:55.842632 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5003e334-4ae4-42e2-a233-df6f10489c69 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:55.846132 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45635de-bb44-4fc2-84d7-b97a38ad74a6 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:55.879827 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a4312-4a6b-433d-88b8-78a648b8b57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:55.880404 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a4312-4a6b-433d-88b8-78a648b8b57b None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:55.880686 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a4312-4a6b-433d-88b8-78a648b8b57b None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:55.880992 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a4312-4a6b-433d-88b8-78a648b8b57b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:55.881822 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4a4312-4a6b-433d-88b8-78a648b8b57b None admin] RBAC: Adding query filter params (group.id=2c24848f4d824e949955fab72678075d, role.id=d336dd7b746c4488b33289c00302e4d5, scope.project.id=b6e5cf8b46f34e30a9b900f3cd1507c0) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:55.882132 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4a4312-4a6b-433d-88b8-78a648b8b57b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:55.883569 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4a4312-4a6b-433d-88b8-78a648b8b57b None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:55.889907 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 225/452] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Jun 27 21:02:55 2022] GET /identity/v3/role_assignments?group.id=2c24848f4d824e949955fab72678075d&scope.project.id=b6e5cf8b46f34e30a9b900f3cd1507c0&role.id=d336dd7b746c4488b33289c00302e4d5 => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:55.891231 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:55.947264 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5003e334-4ae4-42e2-a233-df6f10489c69 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:55.987460 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a728b10e-4db5-4594-9549-3205dc97ba28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:55.988351 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728b10e-4db5-4594-9549-3205dc97ba28 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:55.988633 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728b10e-4db5-4594-9549-3205dc97ba28 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:55.988893 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728b10e-4db5-4594-9549-3205dc97ba28 None admin] PATH_INFO: `/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:55.990206 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a728b10e-4db5-4594-9549-3205dc97ba28 None admin] RBAC: Authorizing `identity:get_project(project_id=8fc48df8fb1f4de59d32fefd09a8ecae)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:55.991702 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a728b10e-4db5-4594-9549-3205dc97ba28 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:55.993472 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 228/453] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:02:55 2022] GET /identity/v3/projects/8fc48df8fb1f4de59d32fefd09a8ecae => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:55.999244 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a4312-4a6b-433d-88b8-78a648b8b57b None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:56.108394 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a4312-4a6b-433d-88b8-78a648b8b57b None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:56.142808 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954a2f52-5bd1-49e4-bec3-6f16e92d3762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:56.143590 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a2f52-5bd1-49e4-bec3-6f16e92d3762 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:56.143874 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a2f52-5bd1-49e4-bec3-6f16e92d3762 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:56.144151 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a2f52-5bd1-49e4-bec3-6f16e92d3762 None admin] PATH_INFO: `/v3/groups/11aca19a34c34bc28cf9d777aef5f6ab` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:56.145386 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954a2f52-5bd1-49e4-bec3-6f16e92d3762 None admin] RBAC: Authorizing `identity:get_group(group_id=11aca19a34c34bc28cf9d777aef5f6ab)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:56.146825 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954a2f52-5bd1-49e4-bec3-6f16e92d3762 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:56.148432 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 226/454] 10.222.0.13 () {56 vars in 1180 bytes} [Mon Jun 27 21:02:55 2022] GET /identity/v3/groups/11aca19a34c34bc28cf9d777aef5f6ab => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:56.155443 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a728b10e-4db5-4594-9549-3205dc97ba28 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:56.258022 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a728b10e-4db5-4594-9549-3205dc97ba28 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:56.274745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:56.293671 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc65c6c-ac41-4a97-8217-0ea4dba7310b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:56.294330 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc65c6c-ac41-4a97-8217-0ea4dba7310b None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:56.294614 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc65c6c-ac41-4a97-8217-0ea4dba7310b None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:56.294891 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc65c6c-ac41-4a97-8217-0ea4dba7310b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:56.295727 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc65c6c-ac41-4a97-8217-0ea4dba7310b None admin] RBAC: Adding query filter params (group.id=11aca19a34c34bc28cf9d777aef5f6ab, role.id=e4414bde4df44de8aacb36c585a93e07, scope.project.id=8fc48df8fb1f4de59d32fefd09a8ecae) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:56.296041 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc65c6c-ac41-4a97-8217-0ea4dba7310b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:56.297519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc65c6c-ac41-4a97-8217-0ea4dba7310b None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:56.303587 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 229/455] 10.222.0.13 () {56 vars in 1376 bytes} [Mon Jun 27 21:02:56 2022] GET /identity/v3/role_assignments?group.id=11aca19a34c34bc28cf9d777aef5f6ab&scope.project.id=8fc48df8fb1f4de59d32fefd09a8ecae&role.id=e4414bde4df44de8aacb36c585a93e07 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:02:59.237434 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ff2cb-0e62-4a83-927c-d9b2dea3297a None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:59.237964 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ff2cb-0e62-4a83-927c-d9b2dea3297a None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:59.237964 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5ff2cb-0e62-4a83-927c-d9b2dea3297a None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:59.239750 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 227/456] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:59.245434 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28ef0e-a683-459d-b764-0cc979be2c45 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:59.246505 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28ef0e-a683-459d-b764-0cc979be2c45 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:59.246990 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28ef0e-a683-459d-b764-0cc979be2c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:59.274242 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-fe28ef0e-a683-459d-b764-0cc979be2c45 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:59.275814 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-fe28ef0e-a683-459d-b764-0cc979be2c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:59.366491 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 230/457] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:59.402807 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ff2cb-0e62-4a83-927c-d9b2dea3297a None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:59.438422 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778c557-0d7c-4583-9577-03d582fc5dd2 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:59.439045 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778c557-0d7c-4583-9577-03d582fc5dd2 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:59.439520 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778c557-0d7c-4583-9577-03d582fc5dd2 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:59.441157 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 231/458] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:02:59.445423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:59.448003 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fa20f-37b1-40cc-8c67-f09531c1c058 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:59.448284 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fa20f-37b1-40cc-8c67-f09531c1c058 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:59.449318 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fa20f-37b1-40cc-8c67-f09531c1c058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:59.472970 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-633fa20f-37b1-40cc-8c67-f09531c1c058 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:59.474181 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-633fa20f-37b1-40cc-8c67-f09531c1c058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:59.586329 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5ff2cb-0e62-4a83-927c-d9b2dea3297a None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:59.586760 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 232/459] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:02:59.588327 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1230fbc6-c4ee-497d-a400-76e4cab20eee None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:59.588761 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1230fbc6-c4ee-497d-a400-76e4cab20eee None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:59.589380 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1230fbc6-c4ee-497d-a400-76e4cab20eee None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:59.591240 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 233/460] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:59.598209 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31634ce-2faf-453b-913f-5d4a7051647b None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:59.598677 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31634ce-2faf-453b-913f-5d4a7051647b None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:59.599101 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31634ce-2faf-453b-913f-5d4a7051647b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:59.626573 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-a31634ce-2faf-453b-913f-5d4a7051647b None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:02:59.627761 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a31634ce-2faf-453b-913f-5d4a7051647b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:02:59.644231 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91357e94-3c99-4202-82f8-b3dfa8da7208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:59.645318 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91357e94-3c99-4202-82f8-b3dfa8da7208 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:59.645769 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91357e94-3c99-4202-82f8-b3dfa8da7208 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:59.646188 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91357e94-3c99-4202-82f8-b3dfa8da7208 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:59.658072 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91357e94-3c99-4202-82f8-b3dfa8da7208 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:59.659764 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91357e94-3c99-4202-82f8-b3dfa8da7208 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:59.673281 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:59.678900 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-91357e94-3c99-4202-82f8-b3dfa8da7208 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:02:59.678900 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:59.678900 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:59.678900 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:59.678900 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:59.678900 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:59.678900 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:59.678900 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:59.678900 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:59.678900 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:59.678900 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:59.678900 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:59.678900 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:02:59.678900 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:02:59.678900 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:02:59.679917 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:02:59.679917 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:59.679917 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:59.679917 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:59.679917 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:59.679917 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:59.679917 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:59.679917 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:59.679917 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:59.679917 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:59.679917 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:59.679917 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:59.679917 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:59.679917 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:59.679917 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:59.679917 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:59.679917 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:59.680950 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:02:59.680950 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:02:59.680950 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:02:59.680950 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:02:59.682958 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 228/461] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:02:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:59.684036 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91357e94-3c99-4202-82f8-b3dfa8da7208 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:59.708400 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 234/462] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:02:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:02:59.716956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31634ce-2faf-453b-913f-5d4a7051647b None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:59.790525 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:02:59.811962 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91357e94-3c99-4202-82f8-b3dfa8da7208 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:59.826845 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31634ce-2faf-453b-913f-5d4a7051647b None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:02:59.847433 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b08edb9-6c09-4835-8120-2f20628cc9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:59.848474 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08edb9-6c09-4835-8120-2f20628cc9ab None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:59.849055 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08edb9-6c09-4835-8120-2f20628cc9ab None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:59.849494 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08edb9-6c09-4835-8120-2f20628cc9ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:59.861614 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b08edb9-6c09-4835-8120-2f20628cc9ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:59.863275 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b08edb9-6c09-4835-8120-2f20628cc9ab None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:59.869580 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a29166-fc3c-4545-b556-fef1eaf17dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:02:59.870275 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a29166-fc3c-4545-b556-fef1eaf17dc8 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:59.870773 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a29166-fc3c-4545-b556-fef1eaf17dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:59.871104 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a29166-fc3c-4545-b556-fef1eaf17dc8 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:59.871848 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a29166-fc3c-4545-b556-fef1eaf17dc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:02:59.872158 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a29166-fc3c-4545-b556-fef1eaf17dc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:02:59.873666 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a29166-fc3c-4545-b556-fef1eaf17dc8 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:02:59.882970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-3b08edb9-6c09-4835-8120-2f20628cc9ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:02:59.882970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:02:59.882970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:02:59.882970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:02:59.882970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:02:59.882970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:02:59.882970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:02:59.882970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:02:59.882970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:02:59.882970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:02:59.882970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:02:59.882970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:02:59.882970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:02:59.882970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:02:59.882970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:02:59.883867 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:02:59.883867 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:02:59.883867 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:02:59.883867 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:02:59.883867 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:02:59.883867 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:02:59.883867 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:02:59.883867 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:02:59.883867 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:02:59.883867 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:02:59.883867 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:02:59.883867 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:02:59.883867 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:02:59.883867 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:02:59.883867 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:02:59.883867 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:02:59.883867 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:02:59.884893 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:02:59.884893 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:02:59.884893 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:02:59.884893 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:02:59.885857 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 235/463] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:02:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:02:59.887167 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 229/464] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:02:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:02:59.888592 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56868c-a4e1-42a5-be3b-d9918cd4f811 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:02:59.889046 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56868c-a4e1-42a5-be3b-d9918cd4f811 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:02:59.889449 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56868c-a4e1-42a5-be3b-d9918cd4f811 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:02:59.891081 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 230/465] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:02:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:02:59.891182 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a29166-fc3c-4545-b556-fef1eaf17dc8 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:59.892388 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56868c-a4e1-42a5-be3b-d9918cd4f811 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:02:59.929464 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:00.006454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56868c-a4e1-42a5-be3b-d9918cd4f811 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:00.028773 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a29166-fc3c-4545-b556-fef1eaf17dc8 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:00.047090 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:00.055461 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e843e0-a7b3-44fa-8987-30e105bf933a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:00.055461 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e843e0-a7b3-44fa-8987-30e105bf933a None admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:00.055461 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e843e0-a7b3-44fa-8987-30e105bf933a None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:00.055461 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e843e0-a7b3-44fa-8987-30e105bf933a None admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:00.055825 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e843e0-a7b3-44fa-8987-30e105bf933a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:00.056747 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e843e0-a7b3-44fa-8987-30e105bf933a None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:00.073116 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6c871-6d89-4efe-8a26-d2d7899644cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:00.074577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6c871-6d89-4efe-8a26-d2d7899644cc None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:00.074960 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6c871-6d89-4efe-8a26-d2d7899644cc None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:00.075282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6c871-6d89-4efe-8a26-d2d7899644cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:00.079864 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 231/466] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Jun 27 21:02:59 2022] POST /identity/v3/users => generated 246 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:03:00.080897 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e843e0-a7b3-44fa-8987-30e105bf933a None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:00.098617 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb6c871-6d89-4efe-8a26-d2d7899644cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:00.101255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb6c871-6d89-4efe-8a26-d2d7899644cc None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:00.134489 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-efb6c871-6d89-4efe-8a26-d2d7899644cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:00.134489 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:00.134489 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:00.134489 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:00.134489 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:00.134489 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:00.134489 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:00.134489 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:00.134489 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:00.134489 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:00.134489 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:00.134489 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:00.134489 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:00.134489 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:00.134489 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:00.135668 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:00.135668 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:00.135668 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:00.135668 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:00.135668 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:00.135668 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:00.135668 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:00.135668 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:00.135668 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:00.135668 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:00.135668 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:00.135668 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:00.135668 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:00.135668 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:00.135668 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:00.135668 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:00.135668 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:00.136578 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:00.136578 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:00.136578 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:00.136578 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:00.142966 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 236/467] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:02:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:00.142966 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477b8eb-b19b-4a2f-bfed-5492746dce66 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:00.142966 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477b8eb-b19b-4a2f-bfed-5492746dce66 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:00.142966 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477b8eb-b19b-4a2f-bfed-5492746dce66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:00.176305 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-a477b8eb-b19b-4a2f-bfed-5492746dce66 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:00.178835 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a477b8eb-b19b-4a2f-bfed-5492746dce66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:00.234346 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e843e0-a7b3-44fa-8987-30e105bf933a None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:00.278707 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 237/468] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:00.280426 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a477b8eb-b19b-4a2f-bfed-5492746dce66 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:00.283573 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ced25-5b6d-4f97-9d14-ecccaf03d387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:00.285211 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ced25-5b6d-4f97-9d14-ecccaf03d387 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:00.285477 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ced25-5b6d-4f97-9d14-ecccaf03d387 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:00.285758 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ced25-5b6d-4f97-9d14-ecccaf03d387 None admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:00.286518 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ced25-5b6d-4f97-9d14-ecccaf03d387 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:00.286825 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ced25-5b6d-4f97-9d14-ecccaf03d387 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:00.288214 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ced25-5b6d-4f97-9d14-ecccaf03d387 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:00.305794 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:00.314322 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 232/469] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:00.315269 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ced25-5b6d-4f97-9d14-ecccaf03d387 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:00.375903 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:00.407111 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a477b8eb-b19b-4a2f-bfed-5492746dce66 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:00.443210 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ced25-5b6d-4f97-9d14-ecccaf03d387 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:00.444713 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e62294-1345-4cea-8beb-3c65798e80b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:00.445424 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e62294-1345-4cea-8beb-3c65798e80b5 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:00.445714 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e62294-1345-4cea-8beb-3c65798e80b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:00.445994 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e62294-1345-4cea-8beb-3c65798e80b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:00.446743 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e62294-1345-4cea-8beb-3c65798e80b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:00.447054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e62294-1345-4cea-8beb-3c65798e80b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:00.448870 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e62294-1345-4cea-8beb-3c65798e80b5 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:00.462020 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 238/470] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:00.467433 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e62294-1345-4cea-8beb-3c65798e80b5 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:00.487488 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857dd0e5-1545-4612-90bb-e56211bc48b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:00.487488 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857dd0e5-1545-4612-90bb-e56211bc48b9 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:00.487488 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857dd0e5-1545-4612-90bb-e56211bc48b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:00.487858 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857dd0e5-1545-4612-90bb-e56211bc48b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:00.498903 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857dd0e5-1545-4612-90bb-e56211bc48b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:00.500475 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857dd0e5-1545-4612-90bb-e56211bc48b9 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:00.522627 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-857dd0e5-1545-4612-90bb-e56211bc48b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:00.522627 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:00.522627 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:00.522627 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:00.522627 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:00.522627 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:00.522627 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:00.522627 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:00.522627 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:00.522627 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:00.522627 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:00.522627 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:00.522627 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:00.522627 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:00.522627 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:00.523644 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:00.523644 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:00.523644 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:00.523644 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:00.523644 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:00.523644 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:00.523644 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:00.523644 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:00.523644 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:00.523644 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:00.523644 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:00.523644 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:00.523644 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:00.523644 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:00.523644 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:00.523644 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:00.523644 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:00.524546 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:00.524546 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:00.524546 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:00.524546 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:00.524546 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 233/471] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:00.525512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21fe4b-9a31-4733-8cd9-6ea5d8797ee7 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:00.525512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21fe4b-9a31-4733-8cd9-6ea5d8797ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:00.525786 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21fe4b-9a31-4733-8cd9-6ea5d8797ee7 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:00.527628 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 234/472] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:00.528375 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21fe4b-9a31-4733-8cd9-6ea5d8797ee7 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:00.572349 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:00.605138 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e62294-1345-4cea-8beb-3c65798e80b5 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:00.649927 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b4f497-bf50-4abc-867d-33e08deecdaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:00.650914 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4f497-bf50-4abc-867d-33e08deecdaa None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:00.651403 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4f497-bf50-4abc-867d-33e08deecdaa None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:00.651969 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4f497-bf50-4abc-867d-33e08deecdaa None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:00.653138 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b4f497-bf50-4abc-867d-33e08deecdaa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:00.654599 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b4f497-bf50-4abc-867d-33e08deecdaa None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:00.663562 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:00.670192 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21fe4b-9a31-4733-8cd9-6ea5d8797ee7 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:00.681689 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 239/473] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Jun 27 21:03:00 2022] POST /identity/v3/users => generated 248 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:03:00.681689 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b4f497-bf50-4abc-867d-33e08deecdaa None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:00.715205 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84344da1-3eb7-4eee-bb46-f894b5555f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:00.716310 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84344da1-3eb7-4eee-bb46-f894b5555f91 None admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:00.716804 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84344da1-3eb7-4eee-bb46-f894b5555f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:00.717331 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84344da1-3eb7-4eee-bb46-f894b5555f91 None admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:00.718333 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84344da1-3eb7-4eee-bb46-f894b5555f91 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:00.720148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84344da1-3eb7-4eee-bb46-f894b5555f91 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:00.745592 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 235/474] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Jun 27 21:03:00 2022] POST /identity/v3/users => generated 248 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:03:00.747535 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9ea8d-e738-4fd6-8a8d-53f2b9afc3de None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:00.748055 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9ea8d-e738-4fd6-8a8d-53f2b9afc3de None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:00.748698 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9ea8d-e738-4fd6-8a8d-53f2b9afc3de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:00.770347 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-4fb9ea8d-e738-4fd6-8a8d-53f2b9afc3de None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:00.772229 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-4fb9ea8d-e738-4fd6-8a8d-53f2b9afc3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:00.810181 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b4f497-bf50-4abc-867d-33e08deecdaa None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:00.844999 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:00.859373 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 236/475] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:00.869990 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cfe60e-6dad-4978-b6d0-e15efb3d2c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:00.872985 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cfe60e-6dad-4978-b6d0-e15efb3d2c12 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:00.872985 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cfe60e-6dad-4978-b6d0-e15efb3d2c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:00.872985 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cfe60e-6dad-4978-b6d0-e15efb3d2c12 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:00.872985 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cfe60e-6dad-4978-b6d0-e15efb3d2c12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:00.872985 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cfe60e-6dad-4978-b6d0-e15efb3d2c12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:00.875459 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cfe60e-6dad-4978-b6d0-e15efb3d2c12 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:00.888593 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9ea8d-e738-4fd6-8a8d-53f2b9afc3de None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:00.901095 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 240/476] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:00.907147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cfe60e-6dad-4978-b6d0-e15efb3d2c12 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:01.020353 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:01.046007 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9ea8d-e738-4fd6-8a8d-53f2b9afc3de None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:01.068246 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cfe60e-6dad-4978-b6d0-e15efb3d2c12 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:01.096546 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ad4aa-f43b-4feb-96f7-cd1522bc1e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:01.097288 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ad4aa-f43b-4feb-96f7-cd1522bc1e3a None admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:01.097568 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ad4aa-f43b-4feb-96f7-cd1522bc1e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:01.097854 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ad4aa-f43b-4feb-96f7-cd1522bc1e3a None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:01.098678 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43ad4aa-f43b-4feb-96f7-cd1522bc1e3a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:01.100154 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43ad4aa-f43b-4feb-96f7-cd1522bc1e3a None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:01.109242 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.sql.core [None req-c43ad4aa-f43b-4feb-96f7-cd1522bc1e3a None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 27 21:03:01.109242 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 27 21:03:01.109242 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [parameters: {'id': '995ab6508adf42d2a92bf636a4cf9131', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 27 21:03:01.109242 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80389) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 27 21:03:01.114886 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a39708-a3c3-405c-8ed7-cac8c52b06a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:01.115630 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a39708-a3c3-405c-8ed7-cac8c52b06a4 None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:01.115975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a39708-a3c3-405c-8ed7-cac8c52b06a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:01.116251 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a39708-a3c3-405c-8ed7-cac8c52b06a4 None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:01.117062 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a39708-a3c3-405c-8ed7-cac8c52b06a4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:01.118693 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a39708-a3c3-405c-8ed7-cac8c52b06a4 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:01.132499 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-c43ad4aa-f43b-4feb-96f7-cd1522bc1e3a None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 27 21:03:01.132499 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:01.132499 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 27 21:03:01.132499 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 27 21:03:01.132499 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 27 21:03:01.132499 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 27 21:03:01.132499 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 27 21:03:01.132499 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application result = self._query(query) Jun 27 21:03:01.132499 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 27 21:03:01.132499 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application conn.query(q) Jun 27 21:03:01.132499 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 27 21:03:01.132499 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 27 21:03:01.132499 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 27 21:03:01.132499 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application result.read() Jun 27 21:03:01.132499 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 27 21:03:01.133547 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 27 21:03:01.133547 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 27 21:03:01.133547 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 27 21:03:01.133547 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 27 21:03:01.133547 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 27 21:03:01.133547 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 27 21:03:01.133547 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 27 21:03:01.133547 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 27 21:03:01.133547 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:01.133547 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 27 21:03:01.133547 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:01.133547 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:01.133547 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 27 21:03:01.133547 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 27 21:03:01.133547 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 27 21:03:01.133547 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return ref.to_dict() Jun 27 21:03:01.133547 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 27 21:03:01.133547 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application next(self.gen) Jun 27 21:03:01.134447 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 27 21:03:01.134447 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application yield resource Jun 27 21:03:01.134447 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 27 21:03:01.134447 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application next(self.gen) Jun 27 21:03:01.134447 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 27 21:03:01.134447 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application self.session.rollback() Jun 27 21:03:01.134447 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 27 21:03:01.134447 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application self.force_reraise() Jun 27 21:03:01.134447 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 27 21:03:01.134447 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise self.value Jun 27 21:03:01.134447 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 27 21:03:01.134447 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 27 21:03:01.134447 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 27 21:03:01.134447 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application session.commit() Jun 27 21:03:01.134447 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 27 21:03:01.134447 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 27 21:03:01.134447 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 27 21:03:01.135302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application self._prepare_impl() Jun 27 21:03:01.135302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 27 21:03:01.135302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application self.session.flush() Jun 27 21:03:01.135302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 27 21:03:01.135302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application self._flush(objects) Jun 27 21:03:01.135302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 27 21:03:01.135302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 27 21:03:01.135302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 27 21:03:01.135302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application compat.raise_( Jun 27 21:03:01.135302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 27 21:03:01.135302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception Jun 27 21:03:01.135302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 27 21:03:01.135302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application flush_context.execute() Jun 27 21:03:01.135302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 27 21:03:01.135302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rec.execute(self) Jun 27 21:03:01.135302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 27 21:03:01.135302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 27 21:03:01.142051 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 27 21:03:01.142051 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 27 21:03:01.142051 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 27 21:03:01.142051 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 27 21:03:01.142051 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 27 21:03:01.142051 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 27 21:03:01.142051 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 27 21:03:01.142051 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 27 21:03:01.142051 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 27 21:03:01.142051 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 27 21:03:01.142051 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 27 21:03:01.142051 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 27 21:03:01.142051 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 27 21:03:01.142051 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 27 21:03:01.142051 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 27 21:03:01.142051 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception Jun 27 21:03:01.144004 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 27 21:03:01.144004 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 27 21:03:01.144004 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 27 21:03:01.144004 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 27 21:03:01.144004 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 27 21:03:01.144004 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application result = self._query(query) Jun 27 21:03:01.144004 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 27 21:03:01.144004 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application conn.query(q) Jun 27 21:03:01.144004 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 27 21:03:01.144004 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 27 21:03:01.144004 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 27 21:03:01.144004 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application result.read() Jun 27 21:03:01.144004 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 27 21:03:01.144004 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 27 21:03:01.144004 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 27 21:03:01.144004 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 27 21:03:01.145987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 27 21:03:01.145987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 27 21:03:01.145987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 27 21:03:01.145987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 27 21:03:01.145987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 27 21:03:01.145987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 27 21:03:01.145987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [parameters: {'id': '995ab6508adf42d2a92bf636a4cf9131', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 27 21:03:01.145987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 27 21:03:01.145987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:01.145987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:03:01.145987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:01.145987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:01.145987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:01.145987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:01.145987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:01.145987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:01.147981 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:01.147981 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:01.147981 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:01.147981 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:01.147981 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:01.147981 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:01.147981 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 27 21:03:01.147981 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 27 21:03:01.147981 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:01.147981 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:01.147981 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 27 21:03:01.147981 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 27 21:03:01.147981 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 27 21:03:01.147981 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 27 21:03:01.147981 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 27 21:03:01.147981 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:01.151717 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 237/477] 10.222.0.13 () {60 vars in 1131 bytes} [Mon Jun 27 21:03:00 2022] POST /identity/v3/roles => generated 144 bytes in 247 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:01.151717 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ad4aa-f43b-4feb-96f7-cd1522bc1e3a None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:01.169614 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 241/478] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Jun 27 21:03:00 2022] POST /identity/v3/users => generated 249 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:03:01.293609 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ad4aa-f43b-4feb-96f7-cd1522bc1e3a None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:01.337275 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9216b028-a616-47ae-95b2-8aab6e7653e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:01.338461 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216b028-a616-47ae-95b2-8aab6e7653e4 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:01.339148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216b028-a616-47ae-95b2-8aab6e7653e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:01.340425 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9216b028-a616-47ae-95b2-8aab6e7653e4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:01.353532 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9216b028-a616-47ae-95b2-8aab6e7653e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:01.363261 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-9216b028-a616-47ae-95b2-8aab6e7653e4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:03:01.363261 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:01.363261 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:01.363261 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:01.363261 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:01.363261 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:01.363261 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:01.363261 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:01.363261 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:01.363261 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:01.363261 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:01.363261 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:01.363261 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:01.363261 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:01.363261 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:01.363261 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:01.364186 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:01.364186 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:01.364186 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:01.364186 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:01.364186 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:01.364186 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:01.364186 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:01.364186 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:01.364186 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:01.364186 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:01.364186 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:01.364186 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:01.364186 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:01.364186 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:01.364186 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:01.364186 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:01.365011 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:01.365011 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:01.365011 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:01.365011 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:01.365011 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:01.365011 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:01.365011 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:01.365011 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:01.365011 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:01.365011 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:01.365011 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 27 21:03:01.365011 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:01.368006 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 238/479] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 27 21:03:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:01.374936 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a39708-a3c3-405c-8ed7-cac8c52b06a4 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:01.460341 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:01.522882 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a39708-a3c3-405c-8ed7-cac8c52b06a4 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:01.569924 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca0257-a04a-4826-b2d2-83611abc6575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:01.571064 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca0257-a04a-4826-b2d2-83611abc6575 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:01.571535 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca0257-a04a-4826-b2d2-83611abc6575 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:01.571997 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca0257-a04a-4826-b2d2-83611abc6575 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:01.572979 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca0257-a04a-4826-b2d2-83611abc6575 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:01.573568 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca0257-a04a-4826-b2d2-83611abc6575 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:01.575769 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca0257-a04a-4826-b2d2-83611abc6575 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:01.589221 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 242/480] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:02.622585 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1baca-f32d-4f2e-8814-41a20b1ed65a None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:02.623117 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1baca-f32d-4f2e-8814-41a20b1ed65a None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:02.623117 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1baca-f32d-4f2e-8814-41a20b1ed65a None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:02.624841 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 239/481] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:03:02.630300 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f18e85-16be-4d48-afcc-4afdae8f547b None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:02.630718 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f18e85-16be-4d48-afcc-4afdae8f547b None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:02.630895 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f18e85-16be-4d48-afcc-4afdae8f547b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:02.657963 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-f9f18e85-16be-4d48-afcc-4afdae8f547b None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:02.659195 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-f9f18e85-16be-4d48-afcc-4afdae8f547b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:02.757751 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 243/482] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:02.793885 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca1baca-f32d-4f2e-8814-41a20b1ed65a None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:02.817511 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:02.942223 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca1baca-f32d-4f2e-8814-41a20b1ed65a None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:02.991474 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07887263-03a0-4207-8a69-3683df40b1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:02.992291 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07887263-03a0-4207-8a69-3683df40b1f1 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:02.992575 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07887263-03a0-4207-8a69-3683df40b1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:02.992852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07887263-03a0-4207-8a69-3683df40b1f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:03.009761 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07887263-03a0-4207-8a69-3683df40b1f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:03.017894 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-07887263-03a0-4207-8a69-3683df40b1f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:03.017894 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:03.017894 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:03.017894 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:03.017894 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:03.017894 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:03.017894 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:03.017894 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:03.017894 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:03.017894 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:03.017894 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:03.017894 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:03.017894 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:03.017894 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:03.017894 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:03.017894 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:03.018767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:03.018767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:03.018767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:03.018767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:03.018767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:03.018767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:03.018767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:03.018767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:03.018767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:03.018767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:03.018767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:03.018767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:03.018767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:03.018767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:03.018767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:03.018767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:03.019590 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:03.019590 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:03.019590 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:03.019590 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:03.019590 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:03.019590 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:03.019590 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:03.019590 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:03.019590 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:03.019590 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:03.019590 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:03.019590 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:03.019590 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 240/483] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 27 21:03:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:03.026097 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f18e85-16be-4d48-afcc-4afdae8f547b None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:03.121346 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:03.252244 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f18e85-16be-4d48-afcc-4afdae8f547b None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:03.281141 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9778d-594a-49e4-bdb8-2c9d80789691 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:03.281462 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9778d-594a-49e4-bdb8-2c9d80789691 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:03.281715 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9778d-594a-49e4-bdb8-2c9d80789691 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:03.283370 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 241/484] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:03:03.289138 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8673bf-3d01-495c-bad1-5396dc454635 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:03.289430 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8673bf-3d01-495c-bad1-5396dc454635 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:03.289705 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8673bf-3d01-495c-bad1-5396dc454635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:03.302779 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ad593-5035-4753-a450-ab368cddc12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:03.303725 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ad593-5035-4753-a450-ab368cddc12f None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:03.304173 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ad593-5035-4753-a450-ab368cddc12f None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:03.304593 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ad593-5035-4753-a450-ab368cddc12f None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:03.305565 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ad593-5035-4753-a450-ab368cddc12f None admin] RBAC: Adding query filter params (name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:03.306030 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ad593-5035-4753-a450-ab368cddc12f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:03.307647 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87ad593-5035-4753-a450-ab368cddc12f None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:03.322051 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 244/485] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 27 21:03:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:03.326188 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-fc8673bf-3d01-495c-bad1-5396dc454635 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:03.327331 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ad593-5035-4753-a450-ab368cddc12f None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:03.329393 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-fc8673bf-3d01-495c-bad1-5396dc454635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:03.445716 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:03.454922 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 242/486] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:03.456333 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b2bd9-16b0-4a4f-87fc-e1e829e5a695 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:03.456621 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b2bd9-16b0-4a4f-87fc-e1e829e5a695 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:03.456899 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b2bd9-16b0-4a4f-87fc-e1e829e5a695 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:03.458488 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 243/487] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:03.464440 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006ab61-cbca-44d7-8c18-7d9bd785c9ad None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:03.465220 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006ab61-cbca-44d7-8c18-7d9bd785c9ad None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:03.465220 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006ab61-cbca-44d7-8c18-7d9bd785c9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:03.483417 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-e006ab61-cbca-44d7-8c18-7d9bd785c9ad None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:03.484677 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-e006ab61-cbca-44d7-8c18-7d9bd785c9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:03.491022 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:03.495125 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ad593-5035-4753-a450-ab368cddc12f None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:03.540724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d88b0d-c986-4d44-b8b3-eddb81fb7319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:03.541916 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d88b0d-c986-4d44-b8b3-eddb81fb7319 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:03.542487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d88b0d-c986-4d44-b8b3-eddb81fb7319 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:03.543754 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d88b0d-c986-4d44-b8b3-eddb81fb7319 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:03.558838 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d88b0d-c986-4d44-b8b3-eddb81fb7319 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:03.561027 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d88b0d-c986-4d44-b8b3-eddb81fb7319 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:03.573117 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 244/488] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:03:03.574778 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e006ab61-cbca-44d7-8c18-7d9bd785c9ad None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:03.583037 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-30d88b0d-c986-4d44-b8b3-eddb81fb7319 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:03.583037 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:03.583037 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:03.583037 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:03.583037 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:03.583037 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:03.583037 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:03.583037 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:03.583037 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:03.583037 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:03.583037 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:03.583037 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:03.583037 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:03.583037 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:03.583037 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:03.584014 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:03.584014 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:03.584014 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:03.584014 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:03.584014 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:03.584014 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:03.584014 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:03.584014 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:03.584014 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:03.584014 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:03.584014 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:03.584014 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:03.584014 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:03.584014 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:03.584014 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:03.584014 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:03.584014 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:03.584848 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:03.584848 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:03.584848 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:03.584848 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:03.587230 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 245/489] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:03.593764 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d88b0d-c986-4d44-b8b3-eddb81fb7319 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:03.715768 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e006ab61-cbca-44d7-8c18-7d9bd785c9ad None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:03.739435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d88b0d-c986-4d44-b8b3-eddb81fb7319 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:03.756975 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:03.766616 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ffb951-66fb-4eeb-aed8-7d20ca6c9ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:03.767701 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ffb951-66fb-4eeb-aed8-7d20ca6c9ab2 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:03.767984 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ffb951-66fb-4eeb-aed8-7d20ca6c9ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:03.768682 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ffb951-66fb-4eeb-aed8-7d20ca6c9ab2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:03.776503 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce2ff0-e6b2-45e3-9899-5fc166e6406f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:03.777576 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce2ff0-e6b2-45e3-9899-5fc166e6406f None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:03.777841 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce2ff0-e6b2-45e3-9899-5fc166e6406f None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:03.778182 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce2ff0-e6b2-45e3-9899-5fc166e6406f None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:03.779012 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ce2ff0-e6b2-45e3-9899-5fc166e6406f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:03.779376 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ce2ff0-e6b2-45e3-9899-5fc166e6406f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:03.780835 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ce2ff0-e6b2-45e3-9899-5fc166e6406f None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:03.783264 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ffb951-66fb-4eeb-aed8-7d20ca6c9ab2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:03.793665 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-d8ffb951-66fb-4eeb-aed8-7d20ca6c9ab2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:03.793665 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:03.793665 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:03.793665 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:03.793665 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:03.793665 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:03.793665 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:03.793665 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:03.793665 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:03.793665 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:03.793665 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:03.793665 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:03.793665 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:03.793665 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:03.793665 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:03.793665 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:03.794762 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:03.794762 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:03.794762 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:03.794762 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:03.794762 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:03.794762 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:03.794762 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:03.794762 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:03.794762 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:03.794762 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:03.794762 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:03.794762 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:03.794762 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:03.794762 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:03.794762 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:03.794762 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:03.795747 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:03.795747 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:03.795747 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:03.795747 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:03.795747 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:03.795747 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:03.795747 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:03.795747 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:03.795747 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:03.795747 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:03.795747 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:03.795747 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:03.798133 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 245/490] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 27 21:03:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:03.798968 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 246/491] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:03.799271 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ffb951-66fb-4eeb-aed8-7d20ca6c9ab2 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:03.800015 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce2ff0-e6b2-45e3-9899-5fc166e6406f None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:03.817661 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:03.911214 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce2ff0-e6b2-45e3-9899-5fc166e6406f None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:03.936422 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ffb951-66fb-4eeb-aed8-7d20ca6c9ab2 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:03.948222 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85974470-8a0b-4809-af76-b0e8b55eb1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:03.949017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85974470-8a0b-4809-af76-b0e8b55eb1cf None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:03.949434 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85974470-8a0b-4809-af76-b0e8b55eb1cf None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:03.949693 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85974470-8a0b-4809-af76-b0e8b55eb1cf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:03.958282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85974470-8a0b-4809-af76-b0e8b55eb1cf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:03.959983 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85974470-8a0b-4809-af76-b0e8b55eb1cf None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:03.974925 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc692cc1-8a04-4a9e-ae30-d48f90625f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:03.976070 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc692cc1-8a04-4a9e-ae30-d48f90625f4f None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:03.976590 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc692cc1-8a04-4a9e-ae30-d48f90625f4f None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:03.977363 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc692cc1-8a04-4a9e-ae30-d48f90625f4f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:03.978879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-85974470-8a0b-4809-af76-b0e8b55eb1cf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 21:03:03.978879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:03.978879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:03.978879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:03.978879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:03.978879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:03.978879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:03.978879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:03.978879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:03.978879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:03.978879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:03.978879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:03.978879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:03.978879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:03.978879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:03.978879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:03.980022 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:03.980022 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:03.980022 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:03.980022 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:03.980022 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:03.980022 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:03.980022 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:03.980022 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:03.980022 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:03.980022 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:03.980022 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:03.980022 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:03.980022 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:03.980022 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:03.980022 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:03.980022 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:03.980022 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:03.980022 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:03.981172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:03.981172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:03.981172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:03.981172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:03.981172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:03.981172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:03.981172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:03.981172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:03.981172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 21:03:03.981172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:03.987070 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 247/492] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Jun 27 21:03:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:03.988067 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85974470-8a0b-4809-af76-b0e8b55eb1cf None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:03.992739 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc692cc1-8a04-4a9e-ae30-d48f90625f4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:04.002657 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-fc692cc1-8a04-4a9e-ae30-d48f90625f4f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:04.002657 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:04.002657 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:04.002657 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:04.002657 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:04.002657 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:04.002657 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:04.002657 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:04.002657 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:04.002657 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:04.002657 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:04.002657 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:04.002657 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:04.002657 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:04.002657 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:04.002657 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:04.003871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:04.003871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:04.003871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:04.003871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:04.003871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:04.003871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:04.003871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:04.003871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:04.003871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:04.003871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:04.003871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:04.003871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:04.003871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:04.003871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:04.003871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:04.003871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:04.004894 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:04.004894 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:04.004894 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:04.004894 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:04.004894 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:04.004894 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:04.004894 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:04.004894 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:04.004894 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:04.004894 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:04.004894 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:04.004894 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:04.012353 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 246/493] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 27 21:03:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:04.013558 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc692cc1-8a04-4a9e-ae30-d48f90625f4f None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:04.065421 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:04.078820 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:04.099118 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85974470-8a0b-4809-af76-b0e8b55eb1cf None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:04.135009 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a5b6cf-3873-43c6-bee7-bbcca1b06d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:04.135849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5b6cf-3873-43c6-bee7-bbcca1b06d8b None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:04.136236 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5b6cf-3873-43c6-bee7-bbcca1b06d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:04.136578 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5b6cf-3873-43c6-bee7-bbcca1b06d8b None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:04.136941 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc692cc1-8a04-4a9e-ae30-d48f90625f4f None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:04.137559 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a5b6cf-3873-43c6-bee7-bbcca1b06d8b None admin] RBAC: Adding query filter params (name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:04.137939 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a5b6cf-3873-43c6-bee7-bbcca1b06d8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:04.139425 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a5b6cf-3873-43c6-bee7-bbcca1b06d8b None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:04.148803 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 248/494] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 27 21:03:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:04.149829 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a5b6cf-3873-43c6-bee7-bbcca1b06d8b None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:04.175778 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99be93e-f183-4d0a-807b-aa9faf234d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:04.176804 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99be93e-f183-4d0a-807b-aa9faf234d63 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:04.177339 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99be93e-f183-4d0a-807b-aa9faf234d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:04.177812 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99be93e-f183-4d0a-807b-aa9faf234d63 None admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:04.178829 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99be93e-f183-4d0a-807b-aa9faf234d63 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:04.179306 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99be93e-f183-4d0a-807b-aa9faf234d63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:04.181112 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99be93e-f183-4d0a-807b-aa9faf234d63 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:04.220034 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 247/495] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Jun 27 21:03:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:04.220885 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99be93e-f183-4d0a-807b-aa9faf234d63 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:04.265481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a5b6cf-3873-43c6-bee7-bbcca1b06d8b None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:04.308888 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb851a1-a687-4888-865e-50df6ae87151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:04.309593 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb851a1-a687-4888-865e-50df6ae87151 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:04.309868 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb851a1-a687-4888-865e-50df6ae87151 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:04.310124 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb851a1-a687-4888-865e-50df6ae87151 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:04.310872 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb851a1-a687-4888-865e-50df6ae87151 None admin] RBAC: Adding query filter params (name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:04.311178 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb851a1-a687-4888-865e-50df6ae87151 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:04.312564 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb851a1-a687-4888-865e-50df6ae87151 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:04.314073 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:04.326435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 249/496] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 27 21:03:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:04.326979 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68dfba-caab-41db-b8dd-de3b31d7e51e None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:04.327262 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68dfba-caab-41db-b8dd-de3b31d7e51e None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:04.327533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68dfba-caab-41db-b8dd-de3b31d7e51e None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:04.328957 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 250/497] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:04.329450 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c68dfba-caab-41db-b8dd-de3b31d7e51e None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:04.345394 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99be93e-f183-4d0a-807b-aa9faf234d63 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:04.362971 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:04.391689 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe7d4a7-fdcb-41a4-a603-63fceba0f2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:04.392501 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe7d4a7-fdcb-41a4-a603-63fceba0f2e6 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:04.392786 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe7d4a7-fdcb-41a4-a603-63fceba0f2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:04.393092 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe7d4a7-fdcb-41a4-a603-63fceba0f2e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:04.403656 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe7d4a7-fdcb-41a4-a603-63fceba0f2e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:04.405239 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe7d4a7-fdcb-41a4-a603-63fceba0f2e6 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:04.422910 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-ffe7d4a7-fdcb-41a4-a603-63fceba0f2e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:04.422910 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:04.422910 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:04.422910 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:04.422910 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:04.422910 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:04.422910 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:04.422910 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:04.422910 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:04.422910 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:04.422910 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:04.422910 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:04.422910 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:04.422910 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:04.422910 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:04.423995 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:04.423995 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:04.423995 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:04.423995 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:04.423995 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:04.423995 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:04.423995 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:04.423995 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:04.423995 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:04.423995 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:04.423995 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:04.423995 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:04.423995 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:04.423995 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:04.423995 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:04.423995 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:04.423995 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:04.424855 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:04.424855 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:04.424855 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:04.424855 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:04.424855 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 248/498] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:04.425567 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd7731-9c8b-4fbd-a670-27978e0e8b55 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:04.425709 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd7731-9c8b-4fbd-a670-27978e0e8b55 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:04.425983 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd7731-9c8b-4fbd-a670-27978e0e8b55 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:04.427375 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 249/499] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:04.430268 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389470c-9391-4b91-80a6-9b622b8a68eb None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:04.430561 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389470c-9391-4b91-80a6-9b622b8a68eb None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:04.430844 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389470c-9391-4b91-80a6-9b622b8a68eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:04.439098 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c68dfba-caab-41db-b8dd-de3b31d7e51e None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:04.447387 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-8389470c-9391-4b91-80a6-9b622b8a68eb None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:04.449023 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-8389470c-9391-4b91-80a6-9b622b8a68eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:04.473251 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c631c80f-1e8a-4127-b520-3da564c97664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:04.473862 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631c80f-1e8a-4127-b520-3da564c97664 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:04.474170 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631c80f-1e8a-4127-b520-3da564c97664 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:04.474451 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631c80f-1e8a-4127-b520-3da564c97664 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:04.518103 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 250/500] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:04.518945 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8389470c-9391-4b91-80a6-9b622b8a68eb None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:04.536762 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:04.545305 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c631c80f-1e8a-4127-b520-3da564c97664 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:04.546781 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c631c80f-1e8a-4127-b520-3da564c97664 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:04.632534 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:04.642959 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8389470c-9391-4b91-80a6-9b622b8a68eb None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:04.647327 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 251/501] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:03:04.648010 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c631c80f-1e8a-4127-b520-3da564c97664 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:04.685914 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9cf91d-1b2b-44a2-b36d-22cf9f56e28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:04.687384 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cf91d-1b2b-44a2-b36d-22cf9f56e28e None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:04.687952 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cf91d-1b2b-44a2-b36d-22cf9f56e28e None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:04.688518 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cf91d-1b2b-44a2-b36d-22cf9f56e28e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:04.703314 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9cf91d-1b2b-44a2-b36d-22cf9f56e28e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:04.705139 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9cf91d-1b2b-44a2-b36d-22cf9f56e28e None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:04.724909 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-df9cf91d-1b2b-44a2-b36d-22cf9f56e28e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:04.724909 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:04.724909 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:04.724909 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:04.724909 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:04.724909 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:04.724909 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:04.724909 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:04.724909 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:04.724909 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:04.724909 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:04.724909 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:04.724909 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:04.724909 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:04.724909 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:04.726566 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:04.726566 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:04.726566 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:04.726566 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:04.726566 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:04.726566 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:04.726566 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:04.726566 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:04.726566 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:04.726566 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:04.726566 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:04.726566 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:04.726566 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:04.726566 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:04.726566 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:04.726566 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:04.726566 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:04.728015 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:04.728015 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:04.728015 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:04.728015 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:04.728015 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 251/502] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:04.728015 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a218812-229a-42ea-a7f9-92adedf83794 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:04.728332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a218812-229a-42ea-a7f9-92adedf83794 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:04.728332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a218812-229a-42ea-a7f9-92adedf83794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:04.747027 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-3a218812-229a-42ea-a7f9-92adedf83794 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:04.748273 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-3a218812-229a-42ea-a7f9-92adedf83794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:04.753261 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:04.759532 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c631c80f-1e8a-4127-b520-3da564c97664 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:04.794899 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803639ef-ab6b-4720-a4ee-f893081f163e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:04.795944 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803639ef-ab6b-4720-a4ee-f893081f163e None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:04.796241 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803639ef-ab6b-4720-a4ee-f893081f163e None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:04.796501 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803639ef-ab6b-4720-a4ee-f893081f163e None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:04.797256 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803639ef-ab6b-4720-a4ee-f893081f163e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:04.797554 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803639ef-ab6b-4720-a4ee-f893081f163e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:04.798938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803639ef-ab6b-4720-a4ee-f893081f163e None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:04.810652 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 252/503] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:04.811454 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803639ef-ab6b-4720-a4ee-f893081f163e None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:04.814625 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 252/504] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:04.815462 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a218812-229a-42ea-a7f9-92adedf83794 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:04.886960 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:04.919563 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803639ef-ab6b-4720-a4ee-f893081f163e None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:04.920954 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a218812-229a-42ea-a7f9-92adedf83794 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:04.953312 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e894794a-0c08-4510-9488-a447d3cf42e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:04.954114 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894794a-0c08-4510-9488-a447d3cf42e9 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:04.954391 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894794a-0c08-4510-9488-a447d3cf42e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:04.954658 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894794a-0c08-4510-9488-a447d3cf42e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:04.960612 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8648e7fc-9516-429d-8c3e-af7332de2065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:04.961462 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8648e7fc-9516-429d-8c3e-af7332de2065 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:04.961729 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8648e7fc-9516-429d-8c3e-af7332de2065 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:04.962003 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8648e7fc-9516-429d-8c3e-af7332de2065 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:04.965966 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e894794a-0c08-4510-9488-a447d3cf42e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:04.973740 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8648e7fc-9516-429d-8c3e-af7332de2065 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:04.973919 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-e894794a-0c08-4510-9488-a447d3cf42e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:04.973919 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:04.973919 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:04.973919 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:04.973919 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:04.973919 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:04.973919 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:04.973919 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:04.973919 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:04.973919 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:04.973919 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:04.973919 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:04.973919 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:04.973919 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:04.973919 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:04.973919 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:04.974789 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:04.974789 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:04.974789 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:04.974789 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:04.974789 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:04.974789 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:04.974789 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:04.974789 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:04.974789 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:04.974789 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:04.974789 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:04.974789 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:04.974789 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:04.974789 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:04.974789 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:04.974789 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:04.975596 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:04.975596 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:04.975596 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:04.975596 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:04.975596 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:04.975596 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:04.975596 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:04.975596 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:04.975596 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:04.975596 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:04.975596 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:04.975596 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:04.975596 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 253/505] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 27 21:03:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:04.976248 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8648e7fc-9516-429d-8c3e-af7332de2065 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:04.976315 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e894794a-0c08-4510-9488-a447d3cf42e9 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:04.998222 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-8648e7fc-9516-429d-8c3e-af7332de2065 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:04.998222 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:04.998222 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:04.998222 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:04.998222 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:04.998222 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:04.998222 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:04.998222 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:04.998222 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:04.998222 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:04.998222 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:04.998222 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:04.998222 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:04.998222 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:04.998222 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:04.999232 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:04.999232 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:04.999232 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:04.999232 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:04.999232 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:04.999232 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:04.999232 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:04.999232 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:04.999232 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:04.999232 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:04.999232 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:04.999232 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:04.999232 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:04.999232 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:04.999232 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:04.999232 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:04.999232 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:05.000275 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:05.000275 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:05.000275 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:05.000275 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:05.000580 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:05.000661 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 253/506] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:05.001011 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8648e7fc-9516-429d-8c3e-af7332de2065 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:05.087467 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e894794a-0c08-4510-9488-a447d3cf42e9 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:05.110992 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8648e7fc-9516-429d-8c3e-af7332de2065 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:05.123695 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3acbc1-3e5c-47bf-898f-bf466b57485f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:05.124382 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3acbc1-3e5c-47bf-898f-bf466b57485f None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:05.124663 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3acbc1-3e5c-47bf-898f-bf466b57485f None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:05.124951 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3acbc1-3e5c-47bf-898f-bf466b57485f None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:05.125701 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3acbc1-3e5c-47bf-898f-bf466b57485f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:05.126007 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3acbc1-3e5c-47bf-898f-bf466b57485f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:05.127403 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3acbc1-3e5c-47bf-898f-bf466b57485f None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:05.140797 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 254/507] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:05.141665 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3acbc1-3e5c-47bf-898f-bf466b57485f None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:05.147381 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:05.161370 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e7f3b-bbcf-49bd-8159-7bfd2e30dd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:05.162177 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e7f3b-bbcf-49bd-8159-7bfd2e30dd80 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:05.162453 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e7f3b-bbcf-49bd-8159-7bfd2e30dd80 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:05.162723 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e7f3b-bbcf-49bd-8159-7bfd2e30dd80 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:05.170285 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65e7f3b-bbcf-49bd-8159-7bfd2e30dd80 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:05.171732 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65e7f3b-bbcf-49bd-8159-7bfd2e30dd80 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:05.188058 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-b65e7f3b-bbcf-49bd-8159-7bfd2e30dd80 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 27 21:03:05.188058 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:05.188058 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:05.188058 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:05.188058 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:05.188058 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:05.188058 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:05.188058 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:05.188058 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:05.188058 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:05.188058 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:05.188058 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:05.188058 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:05.188058 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:05.188058 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:05.188058 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:05.189127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:05.189127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:05.189127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:05.189127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:05.189127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:05.189127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:05.189127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:05.189127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:05.189127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:05.189127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:05.189127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:05.189127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:05.189127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:05.189127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:05.189127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:05.189127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:05.189127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:05.189127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:05.190034 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:05.190034 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:05.190034 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:05.190034 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:05.190034 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:05.190034 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:05.190034 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:05.190034 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:05.190034 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 27 21:03:05.190034 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:05.190034 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 254/508] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:03:04 2022] GET /identity/v3/users/glance => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:05.190596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e7f3b-bbcf-49bd-8159-7bfd2e30dd80 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:05.229423 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:05.260431 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3acbc1-3e5c-47bf-898f-bf466b57485f None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:05.291013 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e7f3b-bbcf-49bd-8159-7bfd2e30dd80 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:05.295444 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0103d042-47d5-4f0d-8dbe-ff0d68f183a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:05.296245 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103d042-47d5-4f0d-8dbe-ff0d68f183a0 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:05.296522 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103d042-47d5-4f0d-8dbe-ff0d68f183a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:05.296791 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0103d042-47d5-4f0d-8dbe-ff0d68f183a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:05.307283 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0103d042-47d5-4f0d-8dbe-ff0d68f183a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:05.308908 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0103d042-47d5-4f0d-8dbe-ff0d68f183a0 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:05.328221 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f58483-4baa-4eee-868b-713c8e167bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:05.329240 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f58483-4baa-4eee-868b-713c8e167bce None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:05.329352 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-0103d042-47d5-4f0d-8dbe-ff0d68f183a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:05.329352 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:05.329352 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:05.329352 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:05.329352 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:05.329352 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:05.329352 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:05.329352 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:05.329352 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:05.329352 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:05.329352 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:05.329352 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:05.329352 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:05.329352 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:05.329352 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:05.330385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:05.330385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:05.330385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:05.330385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:05.330385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:05.330385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:05.330385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:05.330385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:05.330385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:05.330385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:05.330385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:05.330385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:05.330385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:05.330385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:05.330385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:05.330385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:05.330385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:05.331565 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:05.331565 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:05.331565 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:05.331565 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:05.331786 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f58483-4baa-4eee-868b-713c8e167bce None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:05.331786 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f58483-4baa-4eee-868b-713c8e167bce None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:05.331786 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f58483-4baa-4eee-868b-713c8e167bce None admin] RBAC: Adding query filter params (name=service) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:05.331979 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 255/509] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:05.332076 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f58483-4baa-4eee-868b-713c8e167bce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:05.332157 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0103d042-47d5-4f0d-8dbe-ff0d68f183a0 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:05.332335 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f58483-4baa-4eee-868b-713c8e167bce None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:05.345904 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 255/510] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 27 21:03:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:05.346744 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f58483-4baa-4eee-868b-713c8e167bce None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:05.399559 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:05.449281 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0103d042-47d5-4f0d-8dbe-ff0d68f183a0 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:05.457390 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f58483-4baa-4eee-868b-713c8e167bce None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:05.477487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:05.486039 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21554d6-1309-49d8-b687-d5cc2226e46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:05.486709 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21554d6-1309-49d8-b687-d5cc2226e46d None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:05.486985 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21554d6-1309-49d8-b687-d5cc2226e46d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:05.487260 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21554d6-1309-49d8-b687-d5cc2226e46d None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:05.487976 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21554d6-1309-49d8-b687-d5cc2226e46d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:05.488275 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21554d6-1309-49d8-b687-d5cc2226e46d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:05.489675 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21554d6-1309-49d8-b687-d5cc2226e46d None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:05.494572 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b348ddff-35f2-4012-bae5-9e4d0206abc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:05.495558 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348ddff-35f2-4012-bae5-9e4d0206abc1 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:05.496066 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348ddff-35f2-4012-bae5-9e4d0206abc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:05.496066 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b348ddff-35f2-4012-bae5-9e4d0206abc1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:05.500985 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 256/511] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:05.501806 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21554d6-1309-49d8-b687-d5cc2226e46d None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:05.504406 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b348ddff-35f2-4012-bae5-9e4d0206abc1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:05.506765 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b348ddff-35f2-4012-bae5-9e4d0206abc1 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:05.522089 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-b348ddff-35f2-4012-bae5-9e4d0206abc1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 21:03:05.522089 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:05.522089 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:05.522089 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:05.522089 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:05.522089 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:05.522089 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:05.522089 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:05.522089 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:05.522089 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:05.522089 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:05.522089 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:05.522089 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:05.522089 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:05.522089 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:05.522089 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:05.523104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:05.523104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:05.523104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:05.523104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:05.523104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:05.523104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:05.523104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:05.523104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:05.523104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:05.523104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:05.523104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:05.523104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:05.523104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:05.523104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:05.523104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:05.523104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:05.523104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:05.523104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:05.523989 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:05.523989 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:05.523989 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:05.523989 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:05.523989 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:05.523989 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:05.523989 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:05.523989 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:05.523989 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 21:03:05.523989 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:05.523989 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 256/512] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:03:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:05.524568 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b348ddff-35f2-4012-bae5-9e4d0206abc1 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:05.609923 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21554d6-1309-49d8-b687-d5cc2226e46d None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:05.629611 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b348ddff-35f2-4012-bae5-9e4d0206abc1 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:05.637216 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:05.648023 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddad547-7f19-44a3-a120-96e6cff4de3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:05.648723 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddad547-7f19-44a3-a120-96e6cff4de3a None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:05.649029 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddad547-7f19-44a3-a120-96e6cff4de3a None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:05.649293 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddad547-7f19-44a3-a120-96e6cff4de3a None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:05.650194 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddad547-7f19-44a3-a120-96e6cff4de3a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:05.650414 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddad547-7f19-44a3-a120-96e6cff4de3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:05.651889 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddad547-7f19-44a3-a120-96e6cff4de3a None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:05.665231 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc716f9c-a622-423a-bfa6-ee737d32ff2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:05.665897 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc716f9c-a622-423a-bfa6-ee737d32ff2a None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:05.666182 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc716f9c-a622-423a-bfa6-ee737d32ff2a None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:05.666457 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc716f9c-a622-423a-bfa6-ee737d32ff2a None admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:05.667326 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc716f9c-a622-423a-bfa6-ee737d32ff2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:05.667636 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc716f9c-a622-423a-bfa6-ee737d32ff2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:05.669042 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc716f9c-a622-423a-bfa6-ee737d32ff2a None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:05.681748 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 257/513] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:05.682758 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc716f9c-a622-423a-bfa6-ee737d32ff2a None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:05.688698 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 257/514] 10.222.0.13 () {56 vars in 1137 bytes} [Mon Jun 27 21:03:05 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:05.689618 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddad547-7f19-44a3-a120-96e6cff4de3a None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:05.738084 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:05.786504 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc716f9c-a622-423a-bfa6-ee737d32ff2a None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:05.796302 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddad547-7f19-44a3-a120-96e6cff4de3a None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:05.820810 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd132a19-71a6-4380-9be2-ff92a82858c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:05.821632 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd132a19-71a6-4380-9be2-ff92a82858c7 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:05.821906 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd132a19-71a6-4380-9be2-ff92a82858c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:05.822182 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd132a19-71a6-4380-9be2-ff92a82858c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:05.831197 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea954ac0-69ff-474d-ad15-d2f7edc8d6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:05.832004 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea954ac0-69ff-474d-ad15-d2f7edc8d6ff None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:05.832288 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea954ac0-69ff-474d-ad15-d2f7edc8d6ff None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:05.832562 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea954ac0-69ff-474d-ad15-d2f7edc8d6ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:05.832657 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd132a19-71a6-4380-9be2-ff92a82858c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:05.834161 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd132a19-71a6-4380-9be2-ff92a82858c7 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:05.839612 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea954ac0-69ff-474d-ad15-d2f7edc8d6ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:05.841163 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea954ac0-69ff-474d-ad15-d2f7edc8d6ff None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:05.852197 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-cd132a19-71a6-4380-9be2-ff92a82858c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:05.852197 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:05.852197 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:05.852197 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:05.852197 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:05.852197 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:05.852197 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:05.852197 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:05.852197 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:05.852197 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:05.852197 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:05.852197 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:05.852197 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:05.852197 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:05.852197 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:05.853163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:05.853163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:05.853163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:05.853163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:05.853163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:05.853163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:05.853163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:05.853163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:05.853163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:05.853163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:05.853163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:05.853163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:05.853163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:05.853163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:05.853163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:05.853163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:05.853163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:05.854025 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:05.854025 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:05.854025 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:05.854025 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:05.854025 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 258/515] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:05.854307 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd132a19-71a6-4380-9be2-ff92a82858c7 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:05.855892 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-ea954ac0-69ff-474d-ad15-d2f7edc8d6ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:05.855892 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:05.855892 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:05.855892 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:05.855892 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:05.855892 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:05.855892 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:05.855892 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:05.855892 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:05.855892 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:05.855892 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:05.855892 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:05.855892 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:05.855892 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:05.855892 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:05.856912 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:05.856912 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:05.856912 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:05.856912 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:05.856912 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:05.856912 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:05.856912 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:05.856912 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:05.856912 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:05.856912 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:05.856912 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:05.856912 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:05.856912 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:05.856912 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:05.856912 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:05.856912 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:05.856912 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:05.858763 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:05.858763 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:05.858763 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:05.858763 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:05.858763 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:05.858763 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:05.858763 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:05.858763 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:05.858763 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 258/516] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:05.858763 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea954ac0-69ff-474d-ad15-d2f7edc8d6ff None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:05.880086 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:05.960259 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd132a19-71a6-4380-9be2-ff92a82858c7 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:05.962013 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea954ac0-69ff-474d-ad15-d2f7edc8d6ff None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:05.969555 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:05.998859 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bfa6ff-0ca2-493b-8ac0-ec1df8283774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:05.999119 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1843992-6771-44d8-8594-e5e05b2d64ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:05.999532 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bfa6ff-0ca2-493b-8ac0-ec1df8283774 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:05.999774 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1843992-6771-44d8-8594-e5e05b2d64ea None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:05.999862 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bfa6ff-0ca2-493b-8ac0-ec1df8283774 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:06.000053 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1843992-6771-44d8-8594-e5e05b2d64ea None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:06.000151 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bfa6ff-0ca2-493b-8ac0-ec1df8283774 None admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:06.000324 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1843992-6771-44d8-8594-e5e05b2d64ea None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:06.000890 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bfa6ff-0ca2-493b-8ac0-ec1df8283774 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:06.001068 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1843992-6771-44d8-8594-e5e05b2d64ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:06.001231 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bfa6ff-0ca2-493b-8ac0-ec1df8283774 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:06.002526 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1843992-6771-44d8-8594-e5e05b2d64ea None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:06.002650 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bfa6ff-0ca2-493b-8ac0-ec1df8283774 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:06.021203 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 259/517] 10.222.0.13 () {56 vars in 1137 bytes} [Mon Jun 27 21:03:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:06.022035 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bfa6ff-0ca2-493b-8ac0-ec1df8283774 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:06.033196 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 259/518] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Jun 27 21:03:05 2022] POST /identity/v3/users => generated 247 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:03:06.033196 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1843992-6771-44d8-8594-e5e05b2d64ea None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:06.121355 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:06.148053 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bfa6ff-0ca2-493b-8ac0-ec1df8283774 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:06.162270 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1843992-6771-44d8-8594-e5e05b2d64ea None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:06.190616 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e4def-6748-4752-ace5-e2b9661a5f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:06.191287 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e4def-6748-4752-ace5-e2b9661a5f8c None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:06.191571 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e4def-6748-4752-ace5-e2b9661a5f8c None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:06.191844 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e4def-6748-4752-ace5-e2b9661a5f8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:06.201622 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3634bf32-f2cf-419e-a843-d5565d3e8a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:06.201958 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634bf32-f2cf-419e-a843-d5565d3e8a16 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:06.202233 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634bf32-f2cf-419e-a843-d5565d3e8a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:06.202851 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634bf32-f2cf-419e-a843-d5565d3e8a16 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:06.203236 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3634bf32-f2cf-419e-a843-d5565d3e8a16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:06.203547 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3634bf32-f2cf-419e-a843-d5565d3e8a16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:06.204957 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3634bf32-f2cf-419e-a843-d5565d3e8a16 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:06.219910 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 260/519] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:06.220683 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3634bf32-f2cf-419e-a843-d5565d3e8a16 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:06.253292 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:06.268606 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381e4def-6748-4752-ace5-e2b9661a5f8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:06.270149 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381e4def-6748-4752-ace5-e2b9661a5f8c None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:06.346416 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3634bf32-f2cf-419e-a843-d5565d3e8a16 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:06.372013 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:06.383422 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a21a52-9278-4b12-9948-90435c2662b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:06.384085 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a21a52-9278-4b12-9948-90435c2662b7 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:06.384360 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a21a52-9278-4b12-9948-90435c2662b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:06.384626 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a21a52-9278-4b12-9948-90435c2662b7 None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:06.385448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a21a52-9278-4b12-9948-90435c2662b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:06.385748 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a21a52-9278-4b12-9948-90435c2662b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:06.387298 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a21a52-9278-4b12-9948-90435c2662b7 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:06.399371 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 260/520] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:03:06.400464 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e4def-6748-4752-ace5-e2b9661a5f8c None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:06.408001 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 261/521] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:06.408913 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a21a52-9278-4b12-9948-90435c2662b7 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:06.500017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:06.520430 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e4def-6748-4752-ace5-e2b9661a5f8c None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:06.521221 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a21a52-9278-4b12-9948-90435c2662b7 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:06.559868 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f98785-fa3b-49b5-b738-c13a5d9c6fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:06.560740 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f98785-fa3b-49b5-b738-c13a5d9c6fe8 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:06.561064 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f98785-fa3b-49b5-b738-c13a5d9c6fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:06.561367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f98785-fa3b-49b5-b738-c13a5d9c6fe8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:06.564133 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d98384c-681d-4bf7-b600-d63dd941bb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:06.565381 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98384c-681d-4bf7-b600-d63dd941bb04 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:06.565669 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98384c-681d-4bf7-b600-d63dd941bb04 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:06.566006 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98384c-681d-4bf7-b600-d63dd941bb04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:06.569304 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f98785-fa3b-49b5-b738-c13a5d9c6fe8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:06.570793 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f98785-fa3b-49b5-b738-c13a5d9c6fe8 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:06.587465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-81f98785-fa3b-49b5-b738-c13a5d9c6fe8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 27 21:03:06.587465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:06.587465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:06.587465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:06.587465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:06.587465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:06.587465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:06.587465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:06.587465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:06.587465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:06.587465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:06.587465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:06.587465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:06.587465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:06.587465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:06.587465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:06.588462 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:06.588462 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:06.588462 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:06.588462 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:06.588462 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:06.588462 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:06.588462 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:06.588462 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:06.588462 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:06.588462 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:06.588462 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:06.588462 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:06.588462 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:06.588462 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:06.588462 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:06.588462 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:06.588462 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:06.588462 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:06.589507 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:06.589507 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:06.589507 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:06.589507 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:06.589507 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:06.589507 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:06.589507 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:06.589507 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:06.589507 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 27 21:03:06.589507 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:06.589507 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 262/522] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 27 21:03:06 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:06.590075 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f98785-fa3b-49b5-b738-c13a5d9c6fe8 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:06.634260 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:06.662858 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d98384c-681d-4bf7-b600-d63dd941bb04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:06.664378 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d98384c-681d-4bf7-b600-d63dd941bb04 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:06.714583 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f98785-fa3b-49b5-b738-c13a5d9c6fe8 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:06.753624 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b182b38-1aff-4911-94b6-69974819e344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:06.754337 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b182b38-1aff-4911-94b6-69974819e344 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:06.754620 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b182b38-1aff-4911-94b6-69974819e344 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:06.754891 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b182b38-1aff-4911-94b6-69974819e344 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:06.766399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b182b38-1aff-4911-94b6-69974819e344 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:06.768157 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b182b38-1aff-4911-94b6-69974819e344 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:06.771280 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:06.790230 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 261/523] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:03:06.791903 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d98384c-681d-4bf7-b600-d63dd941bb04 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:06.793405 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-3b182b38-1aff-4911-94b6-69974819e344 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:06.793405 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:06.793405 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:06.793405 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:06.793405 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:06.793405 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:06.793405 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:06.793405 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:06.793405 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:06.793405 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:06.793405 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:06.793405 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:06.793405 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:06.793405 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:06.793405 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:06.794484 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:06.794484 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:06.794484 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:06.794484 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:06.794484 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:06.794484 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:06.794484 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:06.794484 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:06.794484 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:06.794484 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:06.794484 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:06.794484 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:06.794484 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:06.794484 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:06.794484 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:06.794484 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:06.794484 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:06.795713 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:06.795713 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:06.795713 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:06.795713 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:06.796901 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 263/524] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:06.797964 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b182b38-1aff-4911-94b6-69974819e344 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:06.880943 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:06.913859 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d98384c-681d-4bf7-b600-d63dd941bb04 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:06.931249 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b182b38-1aff-4911-94b6-69974819e344 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:06.962561 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac1cf6-e63a-4795-9bb6-12729116ac3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:06.963219 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac1cf6-e63a-4795-9bb6-12729116ac3d None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:06.963501 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac1cf6-e63a-4795-9bb6-12729116ac3d None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:06.963774 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac1cf6-e63a-4795-9bb6-12729116ac3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:06.965206 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ac1cf6-e63a-4795-9bb6-12729116ac3d None admin] RBAC: Adding query filter params (role.id=709aef6b85524eb2a8e9986a2bd9525b, scope.project.id=334438c929c44e489e9a3b93e6d37231, user.id=1366958034d540b4ba2feb421a6afabc) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:06.965528 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ac1cf6-e63a-4795-9bb6-12729116ac3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:06.967042 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ac1cf6-e63a-4795-9bb6-12729116ac3d None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:06.980981 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 262/525] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:03:06 2022] GET /identity/v3/role_assignments?user.id=1366958034d540b4ba2feb421a6afabc&scope.project.id=334438c929c44e489e9a3b93e6d37231&role.id=709aef6b85524eb2a8e9986a2bd9525b => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:06.980981 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac1cf6-e63a-4795-9bb6-12729116ac3d None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:06.984796 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c126efa-73e5-4ae3-a1f8-8c9b1ac2ce5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:06.985525 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c126efa-73e5-4ae3-a1f8-8c9b1ac2ce5e None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:06.985808 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c126efa-73e5-4ae3-a1f8-8c9b1ac2ce5e None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:06.986078 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c126efa-73e5-4ae3-a1f8-8c9b1ac2ce5e None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:06.986950 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c126efa-73e5-4ae3-a1f8-8c9b1ac2ce5e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:06.988860 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c126efa-73e5-4ae3-a1f8-8c9b1ac2ce5e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:06.990410 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c126efa-73e5-4ae3-a1f8-8c9b1ac2ce5e None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:07.026897 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 264/526] 10.222.0.13 () {56 vars in 1139 bytes} [Mon Jun 27 21:03:06 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:07.026897 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c126efa-73e5-4ae3-a1f8-8c9b1ac2ce5e None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:07.092054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:07.120749 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac1cf6-e63a-4795-9bb6-12729116ac3d None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:07.151157 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c126efa-73e5-4ae3-a1f8-8c9b1ac2ce5e None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:07.159791 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19749450-d3f9-465f-a687-257ab9de8dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:07.161242 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19749450-d3f9-465f-a687-257ab9de8dbf None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:07.161791 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19749450-d3f9-465f-a687-257ab9de8dbf None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:07.162287 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19749450-d3f9-465f-a687-257ab9de8dbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:07.172878 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:07.176568 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19749450-d3f9-465f-a687-257ab9de8dbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:07.178630 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19749450-d3f9-465f-a687-257ab9de8dbf None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:07.196692 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77628fc-edc1-4c83-a469-c48cc3767b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:07.197806 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77628fc-edc1-4c83-a469-c48cc3767b68 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:07.198311 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77628fc-edc1-4c83-a469-c48cc3767b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:07.198896 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77628fc-edc1-4c83-a469-c48cc3767b68 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:07.199925 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-19749450-d3f9-465f-a687-257ab9de8dbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:07.199925 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:07.199925 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:07.199925 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:07.199925 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:07.199925 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:07.199925 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:07.199925 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:07.199925 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:07.199925 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:07.199925 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:07.199925 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:07.199925 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:07.199925 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:07.199925 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:07.200912 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:07.200912 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:07.200912 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:07.200912 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:07.200912 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:07.200912 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:07.200912 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:07.200912 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:07.200912 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:07.200912 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:07.200912 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:07.200912 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:07.200912 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:07.200912 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:07.200912 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:07.200912 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:07.200912 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:07.201951 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:07.201951 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:07.201951 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:07.201951 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:07.202560 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77628fc-edc1-4c83-a469-c48cc3767b68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:07.203082 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77628fc-edc1-4c83-a469-c48cc3767b68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:07.204667 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77628fc-edc1-4c83-a469-c48cc3767b68 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:07.205572 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 263/527] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:07.206602 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19749450-d3f9-465f-a687-257ab9de8dbf None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:07.219532 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 265/528] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:07.220714 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77628fc-edc1-4c83-a469-c48cc3767b68 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:07.325464 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19749450-d3f9-465f-a687-257ab9de8dbf None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:07.337819 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77628fc-edc1-4c83-a469-c48cc3767b68 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:07.354922 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:07.365463 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32564e6e-c9c7-42bd-bf36-10e00b9d11aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:07.366012 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32564e6e-c9c7-42bd-bf36-10e00b9d11aa None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:07.366355 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32564e6e-c9c7-42bd-bf36-10e00b9d11aa None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:07.366629 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32564e6e-c9c7-42bd-bf36-10e00b9d11aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:07.381965 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f180025-2915-4674-93b8-5651a68971d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:07.383194 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f180025-2915-4674-93b8-5651a68971d4 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:07.383936 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f180025-2915-4674-93b8-5651a68971d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:07.384419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f180025-2915-4674-93b8-5651a68971d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:07.385401 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f180025-2915-4674-93b8-5651a68971d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:07.385968 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f180025-2915-4674-93b8-5651a68971d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:07.387790 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f180025-2915-4674-93b8-5651a68971d4 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:07.401886 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 266/529] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:07.403070 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f180025-2915-4674-93b8-5651a68971d4 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:07.438557 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:07.450289 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32564e6e-c9c7-42bd-bf36-10e00b9d11aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:07.451756 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32564e6e-c9c7-42bd-bf36-10e00b9d11aa None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:07.522419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f180025-2915-4674-93b8-5651a68971d4 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:07.560974 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9dd2fd-0040-49e0-ad9d-c30493630a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:07.562152 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9dd2fd-0040-49e0-ad9d-c30493630a5d None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:07.562661 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9dd2fd-0040-49e0-ad9d-c30493630a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:07.562766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 264/530] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:03:07.563227 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9dd2fd-0040-49e0-ad9d-c30493630a5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:07.563538 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32564e6e-c9c7-42bd-bf36-10e00b9d11aa None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:07.585822 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9dd2fd-0040-49e0-ad9d-c30493630a5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:07.588977 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9dd2fd-0040-49e0-ad9d-c30493630a5d None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:07.619057 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-ec9dd2fd-0040-49e0-ad9d-c30493630a5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:07.619057 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:07.619057 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:07.619057 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:07.619057 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:07.619057 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:07.619057 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:07.619057 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:07.619057 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:07.619057 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:07.619057 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:07.619057 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:07.619057 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:07.619057 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:07.619057 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:07.619970 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:07.619970 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:07.619970 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:07.619970 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:07.619970 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:07.619970 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:07.619970 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:07.619970 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:07.619970 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:07.619970 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:07.619970 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:07.619970 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:07.619970 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:07.619970 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:07.619970 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:07.619970 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:07.619970 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:07.620888 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:07.620888 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:07.620888 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:07.620888 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:07.620888 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:07.620888 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:07.620888 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:07.620888 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:07.624945 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 267/531] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:07.624945 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9dd2fd-0040-49e0-ad9d-c30493630a5d None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:07.676806 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32564e6e-c9c7-42bd-bf36-10e00b9d11aa None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:07.679601 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:07.701374 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:07.744060 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62047a9-d229-486a-a470-7c010d2ae082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:07.746509 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62047a9-d229-486a-a470-7c010d2ae082 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:07.747032 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62047a9-d229-486a-a470-7c010d2ae082 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:07.747732 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62047a9-d229-486a-a470-7c010d2ae082 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:07.765051 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62047a9-d229-486a-a470-7c010d2ae082 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:07.766088 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9dd2fd-0040-49e0-ad9d-c30493630a5d None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:07.768055 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62047a9-d229-486a-a470-7c010d2ae082 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:07.793340 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-f62047a9-d229-486a-a470-7c010d2ae082 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:07.793340 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:07.793340 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:07.793340 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:07.793340 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:07.793340 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:07.793340 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:07.793340 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:07.793340 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:07.793340 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:07.793340 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:07.793340 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:07.793340 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:07.793340 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:07.793340 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:07.794707 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:07.794707 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:07.794707 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:07.794707 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:07.794707 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:07.794707 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:07.794707 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:07.794707 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:07.794707 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:07.794707 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:07.794707 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:07.794707 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:07.794707 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:07.794707 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:07.794707 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:07.794707 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:07.794707 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:07.795549 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:07.795549 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:07.795549 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:07.795549 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:07.795549 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:07.795549 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:07.795549 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:07.795549 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:07.799361 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 265/532] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:07.800433 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62047a9-d229-486a-a470-7c010d2ae082 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:07.809250 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c36adc-4ca9-40ec-831e-fddb6bd8dc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:07.810190 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c36adc-4ca9-40ec-831e-fddb6bd8dc65 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:07.810564 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c36adc-4ca9-40ec-831e-fddb6bd8dc65 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:07.810925 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c36adc-4ca9-40ec-831e-fddb6bd8dc65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:07.822955 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c36adc-4ca9-40ec-831e-fddb6bd8dc65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:07.824719 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c36adc-4ca9-40ec-831e-fddb6bd8dc65 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:07.844340 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-74c36adc-4ca9-40ec-831e-fddb6bd8dc65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:07.844340 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:07.844340 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:07.844340 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:07.844340 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:07.844340 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:07.844340 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:07.844340 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:07.844340 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:07.844340 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:07.844340 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:07.844340 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:07.844340 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:07.844340 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:07.844340 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:07.846077 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:07.846077 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:07.846077 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:07.846077 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:07.846077 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:07.846077 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:07.846077 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:07.846077 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:07.846077 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:07.846077 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:07.846077 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:07.846077 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:07.846077 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:07.846077 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:07.846077 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:07.846077 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:07.846077 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:07.848100 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:07.848100 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:07.848100 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:07.848100 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:07.852947 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 268/533] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:07.852947 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c36adc-4ca9-40ec-831e-fddb6bd8dc65 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:07.916390 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62047a9-d229-486a-a470-7c010d2ae082 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:07.948793 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:07.959513 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597b03d-9b3a-46fb-af36-7074798a8bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:07.960369 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597b03d-9b3a-46fb-af36-7074798a8bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:07.960697 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597b03d-9b3a-46fb-af36-7074798a8bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:07.961013 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597b03d-9b3a-46fb-af36-7074798a8bb2 None admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:07.961985 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2597b03d-9b3a-46fb-af36-7074798a8bb2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:07.962391 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2597b03d-9b3a-46fb-af36-7074798a8bb2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:07.964376 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2597b03d-9b3a-46fb-af36-7074798a8bb2 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:07.966481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c36adc-4ca9-40ec-831e-fddb6bd8dc65 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:07.991742 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 266/534] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:07.992721 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597b03d-9b3a-46fb-af36-7074798a8bb2 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:08.006431 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f1f380-961e-4963-ac6b-95ee948b7085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:08.007100 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1f380-961e-4963-ac6b-95ee948b7085 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:08.007375 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1f380-961e-4963-ac6b-95ee948b7085 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:08.007645 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1f380-961e-4963-ac6b-95ee948b7085 None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:08.008432 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f1f380-961e-4963-ac6b-95ee948b7085 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:08.008735 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f1f380-961e-4963-ac6b-95ee948b7085 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:08.009150 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:08.011669 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f1f380-961e-4963-ac6b-95ee948b7085 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:08.026573 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 269/535] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:08.027912 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f1f380-961e-4963-ac6b-95ee948b7085 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:08.127460 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597b03d-9b3a-46fb-af36-7074798a8bb2 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:08.164833 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87e99f-9912-4807-915e-d2074dd613ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:08.165586 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87e99f-9912-4807-915e-d2074dd613ea None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:08.166005 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87e99f-9912-4807-915e-d2074dd613ea None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:08.166222 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87e99f-9912-4807-915e-d2074dd613ea None admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:08.167218 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb87e99f-9912-4807-915e-d2074dd613ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:08.167398 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb87e99f-9912-4807-915e-d2074dd613ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:08.168799 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb87e99f-9912-4807-915e-d2074dd613ea None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:08.173972 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f1f380-961e-4963-ac6b-95ee948b7085 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:08.187420 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 267/536] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:08.187420 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87e99f-9912-4807-915e-d2074dd613ea None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:08.216515 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e33c3-b474-4470-837a-6caf0982e44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:08.217581 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e33c3-b474-4470-837a-6caf0982e44d None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:08.218001 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e33c3-b474-4470-837a-6caf0982e44d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:08.218411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e33c3-b474-4470-837a-6caf0982e44d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:08.219480 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312e33c3-b474-4470-837a-6caf0982e44d None admin] RBAC: Adding query filter params (role.id=709aef6b85524eb2a8e9986a2bd9525b, scope.project.id=334438c929c44e489e9a3b93e6d37231, user.id=26cf962d151b4da7a82c33ef23b342e6) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:08.219926 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312e33c3-b474-4470-837a-6caf0982e44d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:08.221687 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312e33c3-b474-4470-837a-6caf0982e44d None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:08.228642 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 270/537] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:03:08 2022] GET /identity/v3/role_assignments?user.id=26cf962d151b4da7a82c33ef23b342e6&scope.project.id=334438c929c44e489e9a3b93e6d37231&role.id=709aef6b85524eb2a8e9986a2bd9525b => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:08.229919 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e33c3-b474-4470-837a-6caf0982e44d None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:08.261492 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:08.316629 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:08.386737 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87e99f-9912-4807-915e-d2074dd613ea None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:08.455559 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b121a82d-c236-470b-adba-1f2c0a574876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:08.456257 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b121a82d-c236-470b-adba-1f2c0a574876 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:08.456537 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b121a82d-c236-470b-adba-1f2c0a574876 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:08.456807 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b121a82d-c236-470b-adba-1f2c0a574876 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:08.458544 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b121a82d-c236-470b-adba-1f2c0a574876 None admin] RBAC: Adding query filter params (role.id=709aef6b85524eb2a8e9986a2bd9525b, scope.project.id=334438c929c44e489e9a3b93e6d37231, user.id=340915e7debe4ce6828bd8a477ed6f16) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:08.458852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b121a82d-c236-470b-adba-1f2c0a574876 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:08.460318 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b121a82d-c236-470b-adba-1f2c0a574876 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:08.468546 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 268/538] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:03:08 2022] GET /identity/v3/role_assignments?user.id=340915e7debe4ce6828bd8a477ed6f16&scope.project.id=334438c929c44e489e9a3b93e6d37231&role.id=709aef6b85524eb2a8e9986a2bd9525b => generated 259 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:08.473115 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e33c3-b474-4470-837a-6caf0982e44d None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:08.497594 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f3dd6-2b5b-4a88-af6d-503222a1dc4c None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:08.497931 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f3dd6-2b5b-4a88-af6d-503222a1dc4c None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:08.498142 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f3dd6-2b5b-4a88-af6d-503222a1dc4c None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:08.499945 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 269/539] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:03:08.505900 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655ef91-e5c2-4855-92fa-a78a88bdcb0e None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:08.506224 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655ef91-e5c2-4855-92fa-a78a88bdcb0e None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:08.506457 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655ef91-e5c2-4855-92fa-a78a88bdcb0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:08.529635 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-6655ef91-e5c2-4855-92fa-a78a88bdcb0e None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:08.530863 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-6655ef91-e5c2-4855-92fa-a78a88bdcb0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:08.537416 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64faf0c2-0df3-4992-9c1e-0d49b3b831ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:08.538228 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64faf0c2-0df3-4992-9c1e-0d49b3b831ae None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:08.538503 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64faf0c2-0df3-4992-9c1e-0d49b3b831ae None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:08.538769 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64faf0c2-0df3-4992-9c1e-0d49b3b831ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:08.552454 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64faf0c2-0df3-4992-9c1e-0d49b3b831ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:08.554004 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64faf0c2-0df3-4992-9c1e-0d49b3b831ae None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:08.579465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-64faf0c2-0df3-4992-9c1e-0d49b3b831ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:08.579465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:08.579465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:08.579465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:08.579465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:08.579465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:08.579465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:08.579465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:08.579465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:08.579465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:08.579465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:08.579465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:08.579465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:08.579465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:08.579465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:08.580395 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:08.580395 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:08.580395 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:08.580395 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:08.580395 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:08.580395 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:08.580395 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:08.580395 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:08.580395 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:08.580395 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:08.580395 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:08.580395 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:08.580395 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:08.580395 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:08.580395 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:08.580395 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:08.580395 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:08.581288 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:08.581288 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:08.581288 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:08.581288 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:08.581288 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:08.581288 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:08.581288 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:08.581288 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:08.586635 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 271/540] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:08.592011 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64faf0c2-0df3-4992-9c1e-0d49b3b831ae None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:08.681192 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 270/541] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:03:08.705965 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6655ef91-e5c2-4855-92fa-a78a88bdcb0e None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:08.746982 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:08.788330 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:08.832072 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6655ef91-e5c2-4855-92fa-a78a88bdcb0e None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:08.852299 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64faf0c2-0df3-4992-9c1e-0d49b3b831ae None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:08.880337 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b6feda-4f54-4a51-80d8-955daed06cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:08.882152 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b6feda-4f54-4a51-80d8-955daed06cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:08.882654 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b6feda-4f54-4a51-80d8-955daed06cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:08.883436 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b6feda-4f54-4a51-80d8-955daed06cd0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:08.896675 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b6feda-4f54-4a51-80d8-955daed06cd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:08.906512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-b5b6feda-4f54-4a51-80d8-955daed06cd0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:08.906512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:08.906512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:08.906512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:08.906512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:08.906512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:08.906512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:08.906512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:08.906512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:08.906512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:08.906512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:08.906512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:08.906512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:08.906512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:08.906512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:08.906512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:08.907467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:08.907467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:08.907467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:08.907467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:08.907467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:08.907467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:08.907467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:08.907467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:08.907467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:08.907467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:08.907467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:08.907467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:08.907467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:08.907467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:08.907467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:08.907467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:08.908261 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:08.908261 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:08.908261 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:08.908261 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:08.908261 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:08.908261 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:08.908261 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:08.908261 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:08.908261 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:08.908261 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:08.908261 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:08.908261 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:08.911383 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 271/542] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 27 21:03:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:08.917198 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b6feda-4f54-4a51-80d8-955daed06cd0 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:08.941741 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88582d7b-cefc-4a9c-8df4-0972b56fce3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:08.942336 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88582d7b-cefc-4a9c-8df4-0972b56fce3e None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:08.942619 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88582d7b-cefc-4a9c-8df4-0972b56fce3e None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:08.942890 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88582d7b-cefc-4a9c-8df4-0972b56fce3e None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:08.943685 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88582d7b-cefc-4a9c-8df4-0972b56fce3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:08.943989 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88582d7b-cefc-4a9c-8df4-0972b56fce3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:08.945492 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88582d7b-cefc-4a9c-8df4-0972b56fce3e None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:08.971429 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 272/543] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:08.986053 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88582d7b-cefc-4a9c-8df4-0972b56fce3e None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:09.059764 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b6feda-4f54-4a51-80d8-955daed06cd0 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:09.088282 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:09.099673 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b225922c-64b5-4153-9f62-ca4016e7fc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:09.100716 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225922c-64b5-4153-9f62-ca4016e7fc49 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:09.101314 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225922c-64b5-4153-9f62-ca4016e7fc49 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:09.102151 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225922c-64b5-4153-9f62-ca4016e7fc49 None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:09.103145 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b225922c-64b5-4153-9f62-ca4016e7fc49 None admin] RBAC: Adding query filter params (name=service) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:09.103655 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b225922c-64b5-4153-9f62-ca4016e7fc49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:09.105417 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b225922c-64b5-4153-9f62-ca4016e7fc49 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:09.114788 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 272/544] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 27 21:03:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:09.122023 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b225922c-64b5-4153-9f62-ca4016e7fc49 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:09.154922 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88582d7b-cefc-4a9c-8df4-0972b56fce3e None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:09.182483 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:09.190904 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f26d16-cb50-490d-88dc-be4efad4350d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:09.191559 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f26d16-cb50-490d-88dc-be4efad4350d None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:09.191888 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f26d16-cb50-490d-88dc-be4efad4350d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:09.192101 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f26d16-cb50-490d-88dc-be4efad4350d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:09.192987 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f26d16-cb50-490d-88dc-be4efad4350d None admin] RBAC: Adding query filter params (role.id=709aef6b85524eb2a8e9986a2bd9525b, scope.project.id=334438c929c44e489e9a3b93e6d37231, user.id=2a4e105858324391b5142d57e34aff9c) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:09.193308 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f26d16-cb50-490d-88dc-be4efad4350d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:09.194879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f26d16-cb50-490d-88dc-be4efad4350d None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:09.201468 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 273/545] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:03:08 2022] GET /identity/v3/role_assignments?user.id=2a4e105858324391b5142d57e34aff9c&scope.project.id=334438c929c44e489e9a3b93e6d37231&role.id=709aef6b85524eb2a8e9986a2bd9525b => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:09.290342 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b225922c-64b5-4153-9f62-ca4016e7fc49 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:09.322791 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a73986-173c-4cd5-94bf-276140e331e7 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:09.323111 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a73986-173c-4cd5-94bf-276140e331e7 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:09.323462 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a73986-173c-4cd5-94bf-276140e331e7 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:09.329346 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 274/546] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:09 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:03:09.329546 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee4288-9342-46d6-b35b-82e2c59cbe8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:09.330237 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee4288-9342-46d6-b35b-82e2c59cbe8d None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:09.330561 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee4288-9342-46d6-b35b-82e2c59cbe8d None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:09.330835 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee4288-9342-46d6-b35b-82e2c59cbe8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:09.330993 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d489e2f-445d-49d0-ba07-ee250e9901b2 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:09.331277 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d489e2f-445d-49d0-ba07-ee250e9901b2 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:09.331559 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d489e2f-445d-49d0-ba07-ee250e9901b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:09.344122 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee4288-9342-46d6-b35b-82e2c59cbe8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:09.345638 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee4288-9342-46d6-b35b-82e2c59cbe8d None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:09.368518 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-08ee4288-9342-46d6-b35b-82e2c59cbe8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:09.368518 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:09.368518 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:09.368518 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:09.368518 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:09.368518 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:09.368518 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:09.368518 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:09.368518 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:09.368518 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:09.368518 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:09.368518 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:09.368518 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:09.368518 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:09.368518 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:09.369472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:09.369472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:09.369472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:09.369472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:09.369472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:09.369472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:09.369472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:09.369472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:09.369472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:09.369472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:09.369472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:09.369472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:09.369472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:09.369472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:09.369472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:09.369472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:09.369472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:09.370366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:09.370366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:09.370366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:09.370366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:09.370366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 273/547] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:09.378166 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-6d489e2f-445d-49d0-ba07-ee250e9901b2 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:09.379426 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-6d489e2f-445d-49d0-ba07-ee250e9901b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:09.382260 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee4288-9342-46d6-b35b-82e2c59cbe8d None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:09.434066 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:09.503703 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee4288-9342-46d6-b35b-82e2c59cbe8d None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:09.544565 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a93c9b9-623d-4cb1-8498-782ed4c1e8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:09.546157 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93c9b9-623d-4cb1-8498-782ed4c1e8ee None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:09.546670 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93c9b9-623d-4cb1-8498-782ed4c1e8ee None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:09.547140 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a93c9b9-623d-4cb1-8498-782ed4c1e8ee None admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:09.548065 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a93c9b9-623d-4cb1-8498-782ed4c1e8ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:09.548571 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a93c9b9-623d-4cb1-8498-782ed4c1e8ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:09.550658 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a93c9b9-623d-4cb1-8498-782ed4c1e8ee None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:09.567431 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 274/548] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:09.575457 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a93c9b9-623d-4cb1-8498-782ed4c1e8ee None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:09.599632 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 275/549] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:09.634212 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d489e2f-445d-49d0-ba07-ee250e9901b2 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:09.749921 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:09.757100 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a93c9b9-623d-4cb1-8498-782ed4c1e8ee None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:09.811789 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce19f899-bbd9-436b-bb0c-cbc938b0e6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:09.813037 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce19f899-bbd9-436b-bb0c-cbc938b0e6de None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:09.813546 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce19f899-bbd9-436b-bb0c-cbc938b0e6de None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:09.814135 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce19f899-bbd9-436b-bb0c-cbc938b0e6de None admin] PATH_INFO: `/v3/users/swift` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:09.822739 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce19f899-bbd9-436b-bb0c-cbc938b0e6de None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:09.824767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce19f899-bbd9-436b-bb0c-cbc938b0e6de None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:09.837839 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d489e2f-445d-49d0-ba07-ee250e9901b2 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:09.848655 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-ce19f899-bbd9-436b-bb0c-cbc938b0e6de None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 27 21:03:09.848655 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:09.848655 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:09.848655 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:09.848655 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:09.848655 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:09.848655 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:09.848655 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:09.848655 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:09.848655 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:09.848655 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:09.848655 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:09.848655 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:09.848655 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:09.848655 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:09.848655 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:09.850463 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:09.850463 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:09.850463 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:09.850463 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:09.850463 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:09.850463 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:09.850463 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:09.850463 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:09.850463 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:09.850463 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:09.850463 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:09.850463 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:09.850463 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:09.850463 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:09.850463 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:09.850463 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:09.850463 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:09.850463 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:09.851524 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:09.851524 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:09.851524 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:09.851524 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:09.851524 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:09.851524 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:09.851524 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:09.851524 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:09.851524 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 27 21:03:09.851524 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:09.854383 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 275/550] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 27 21:03:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:09.862099 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce19f899-bbd9-436b-bb0c-cbc938b0e6de None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:09.870005 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:09.896642 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbec95b-bf95-4fa1-900d-5a0f86087f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:09.898491 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbec95b-bf95-4fa1-900d-5a0f86087f6d None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:09.898627 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbec95b-bf95-4fa1-900d-5a0f86087f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:09.898872 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbec95b-bf95-4fa1-900d-5a0f86087f6d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:09.919998 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbec95b-bf95-4fa1-900d-5a0f86087f6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:09.939925 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-abbec95b-bf95-4fa1-900d-5a0f86087f6d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:09.939925 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:09.939925 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:09.939925 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:09.939925 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:09.939925 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:09.939925 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:09.939925 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:09.939925 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:09.939925 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:09.939925 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:09.939925 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:09.939925 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:09.939925 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:09.939925 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:09.939925 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:09.940996 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:09.940996 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:09.940996 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:09.940996 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:09.940996 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:09.940996 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:09.940996 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:09.940996 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:09.940996 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:09.940996 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:09.940996 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:09.940996 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:09.940996 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:09.940996 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:09.940996 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:09.940996 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:09.941999 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:09.941999 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:09.941999 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:09.941999 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:09.941999 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:09.941999 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:09.941999 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:09.941999 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:09.941999 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:09.941999 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:09.941999 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:09.941999 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:09.942626 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 276/551] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 27 21:03:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:09.947704 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbec95b-bf95-4fa1-900d-5a0f86087f6d None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:10.089195 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbec95b-bf95-4fa1-900d-5a0f86087f6d None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:10.124903 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f762b-2380-40db-8eba-87282e5155e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:10.125695 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f762b-2380-40db-8eba-87282e5155e7 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:10.125982 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f762b-2380-40db-8eba-87282e5155e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:10.126259 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f762b-2380-40db-8eba-87282e5155e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:10.127015 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f762b-2380-40db-8eba-87282e5155e7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:10.127325 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f762b-2380-40db-8eba-87282e5155e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:10.128874 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f762b-2380-40db-8eba-87282e5155e7 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:10.139855 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce19f899-bbd9-436b-bb0c-cbc938b0e6de None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:10.142013 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 277/552] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 27 21:03:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:10.143304 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f762b-2380-40db-8eba-87282e5155e7 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:10.193328 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:10.238108 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a928da6e-c2e9-4f4e-a066-d023ee079496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:10.239194 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928da6e-c2e9-4f4e-a066-d023ee079496 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:10.239484 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928da6e-c2e9-4f4e-a066-d023ee079496 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:10.239762 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928da6e-c2e9-4f4e-a066-d023ee079496 None admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:10.240629 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a928da6e-c2e9-4f4e-a066-d023ee079496 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:10.240957 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a928da6e-c2e9-4f4e-a066-d023ee079496 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:10.242455 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a928da6e-c2e9-4f4e-a066-d023ee079496 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:10.247380 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f762b-2380-40db-8eba-87282e5155e7 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:10.283696 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5192eb8-0083-42ce-9be2-a86c78ad96e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:10.284464 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5192eb8-0083-42ce-9be2-a86c78ad96e4 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:10.284745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5192eb8-0083-42ce-9be2-a86c78ad96e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:10.285093 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5192eb8-0083-42ce-9be2-a86c78ad96e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:10.295353 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 276/553] 10.222.0.13 () {56 vars in 1135 bytes} [Mon Jun 27 21:03:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 434 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:10.296512 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5192eb8-0083-42ce-9be2-a86c78ad96e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:10.298062 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5192eb8-0083-42ce-9be2-a86c78ad96e4 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:10.306022 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a928da6e-c2e9-4f4e-a066-d023ee079496 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:10.317233 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-c5192eb8-0083-42ce-9be2-a86c78ad96e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:10.317233 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:10.317233 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:10.317233 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:10.317233 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:10.317233 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:10.317233 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:10.317233 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:10.317233 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:10.317233 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:10.317233 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:10.317233 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:10.317233 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:10.317233 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:10.317233 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:10.318157 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:10.318157 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:10.318157 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:10.318157 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:10.318157 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:10.318157 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:10.318157 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:10.318157 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:10.318157 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:10.318157 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:10.318157 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:10.318157 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:10.318157 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:10.318157 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:10.318157 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:10.318157 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:10.318157 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:10.319047 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:10.319047 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:10.319047 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:10.319047 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:10.323152 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 278/554] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:10.324655 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5192eb8-0083-42ce-9be2-a86c78ad96e4 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:10.352518 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:10.489960 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5192eb8-0083-42ce-9be2-a86c78ad96e4 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:10.500331 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:10.542136 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a928da6e-c2e9-4f4e-a066-d023ee079496 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:10.545225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c3027c-ee73-4d18-af68-3ba5b8593c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:10.545899 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3027c-ee73-4d18-af68-3ba5b8593c32 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:10.546224 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3027c-ee73-4d18-af68-3ba5b8593c32 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:10.546497 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3027c-ee73-4d18-af68-3ba5b8593c32 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:10.547230 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c3027c-ee73-4d18-af68-3ba5b8593c32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:10.547531 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c3027c-ee73-4d18-af68-3ba5b8593c32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:10.548910 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c3027c-ee73-4d18-af68-3ba5b8593c32 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:10.578355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 279/555] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:10.581993 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c3027c-ee73-4d18-af68-3ba5b8593c32 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:10.615023 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3c611-610e-4829-8197-70808c2ab609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:10.615691 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3c611-610e-4829-8197-70808c2ab609 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:10.615976 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3c611-610e-4829-8197-70808c2ab609 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:10.616250 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3c611-610e-4829-8197-70808c2ab609 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:10.697585 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c3027c-ee73-4d18-af68-3ba5b8593c32 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:10.736401 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6bd7a9-81a7-4e96-92c0-6c908083abf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:10.737279 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6bd7a9-81a7-4e96-92c0-6c908083abf3 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:10.737990 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6bd7a9-81a7-4e96-92c0-6c908083abf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:10.738325 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6bd7a9-81a7-4e96-92c0-6c908083abf3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:10.747698 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6bd7a9-81a7-4e96-92c0-6c908083abf3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:10.749182 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6bd7a9-81a7-4e96-92c0-6c908083abf3 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:10.765470 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-bd6bd7a9-81a7-4e96-92c0-6c908083abf3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 21:03:10.765470 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:10.765470 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:10.765470 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:10.765470 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:10.765470 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:10.765470 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:10.765470 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:10.765470 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:10.765470 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:10.765470 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:10.765470 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:10.765470 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:10.765470 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:10.765470 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:10.765470 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:10.766448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:10.766448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:10.766448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:10.766448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:10.766448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:10.766448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:10.766448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:10.766448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:10.766448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:10.766448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:10.766448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:10.766448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:10.766448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:10.766448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:10.766448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:10.766448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:10.766448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:10.766448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:10.767857 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:10.767857 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:10.767857 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:10.767857 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:10.767857 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:10.767857 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:10.767857 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:10.767857 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:10.767857 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 21:03:10.767857 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:10.771439 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 280/556] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Jun 27 21:03:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:10.772712 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6bd7a9-81a7-4e96-92c0-6c908083abf3 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:10.794922 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:10.845847 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d3c611-610e-4829-8197-70808c2ab609 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:10.847352 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d3c611-610e-4829-8197-70808c2ab609 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:10.876427 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:10.930359 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6bd7a9-81a7-4e96-92c0-6c908083abf3 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:10.980430 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e096382-8b4e-4adb-a369-d68381ab9862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:10.981153 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e096382-8b4e-4adb-a369-d68381ab9862 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:10.981434 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e096382-8b4e-4adb-a369-d68381ab9862 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:10.981707 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e096382-8b4e-4adb-a369-d68381ab9862 None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:10.982554 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e096382-8b4e-4adb-a369-d68381ab9862 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:10.982935 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 277/557] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 682 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 21:03:10.983042 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e096382-8b4e-4adb-a369-d68381ab9862 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:10.984298 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e096382-8b4e-4adb-a369-d68381ab9862 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:10.984415 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75a183-c658-42ea-96f5-ca95c2476978 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:10.984586 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75a183-c658-42ea-96f5-ca95c2476978 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:10.984843 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75a183-c658-42ea-96f5-ca95c2476978 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:10.986121 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 278/558] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:10.990198 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253504b3-2a35-4bac-a953-78f6dc90bb77 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:10.990734 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253504b3-2a35-4bac-a953-78f6dc90bb77 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:10.991014 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253504b3-2a35-4bac-a953-78f6dc90bb77 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:10.992461 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 279/559] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:10.993065 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253504b3-2a35-4bac-a953-78f6dc90bb77 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:11.004566 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 281/560] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Jun 27 21:03:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:11.006315 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04d0a6-a789-4749-aa73-ba83203a8078 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:11.006818 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04d0a6-a789-4749-aa73-ba83203a8078 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:11.007283 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c04d0a6-a789-4749-aa73-ba83203a8078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:11.027287 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-4c04d0a6-a789-4749-aa73-ba83203a8078 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:11.028546 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-4c04d0a6-a789-4749-aa73-ba83203a8078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:11.113585 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253504b3-2a35-4bac-a953-78f6dc90bb77 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:11.117011 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 282/561] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:11.117011 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f17bca3-9baf-4f51-83d9-d2dd7b65b85d None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:11.117011 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f17bca3-9baf-4f51-83d9-d2dd7b65b85d None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:11.117011 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f17bca3-9baf-4f51-83d9-d2dd7b65b85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:11.123997 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:11.140673 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:11.141854 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-9f17bca3-9baf-4f51-83d9-d2dd7b65b85d None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:11.143069 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-9f17bca3-9baf-4f51-83d9-d2dd7b65b85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:11.158228 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f26c8-4fdb-4628-a706-e86f550e1962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:11.159044 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f26c8-4fdb-4628-a706-e86f550e1962 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:11.159327 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f26c8-4fdb-4628-a706-e86f550e1962 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:11.159603 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f26c8-4fdb-4628-a706-e86f550e1962 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:11.171159 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f26c8-4fdb-4628-a706-e86f550e1962 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:11.172704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f26c8-4fdb-4628-a706-e86f550e1962 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:11.192517 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-c95f26c8-4fdb-4628-a706-e86f550e1962 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:11.192517 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:11.192517 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:11.192517 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:11.192517 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:11.192517 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:11.192517 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:11.192517 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:11.192517 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:11.192517 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:11.192517 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:11.192517 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:11.192517 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:11.192517 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:11.192517 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:11.194008 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:11.194008 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:11.194008 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:11.194008 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:11.194008 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:11.194008 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:11.194008 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:11.194008 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:11.194008 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:11.194008 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:11.194008 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:11.194008 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:11.194008 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:11.194008 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:11.194008 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:11.194008 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:11.194008 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:11.195107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:11.195107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:11.195107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:11.195107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:11.195107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 280/562] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:11.195107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f26c8-4fdb-4628-a706-e86f550e1962 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:11.234950 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 283/563] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:11.234950 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f17bca3-9baf-4f51-83d9-d2dd7b65b85d None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:11.322524 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f26c8-4fdb-4628-a706-e86f550e1962 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:11.349912 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f17bca3-9baf-4f51-83d9-d2dd7b65b85d None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:11.361857 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c36dd0e-9299-4744-b9a3-1e3b4c136c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:11.362665 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36dd0e-9299-4744-b9a3-1e3b4c136c54 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:11.362947 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36dd0e-9299-4744-b9a3-1e3b4c136c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:11.363220 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36dd0e-9299-4744-b9a3-1e3b4c136c54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:11.376116 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c36dd0e-9299-4744-b9a3-1e3b4c136c54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:11.377968 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c36dd0e-9299-4744-b9a3-1e3b4c136c54 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:11.396289 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-2c36dd0e-9299-4744-b9a3-1e3b4c136c54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:11.396289 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:11.396289 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:11.396289 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:11.396289 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:11.396289 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:11.396289 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:11.396289 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:11.396289 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:11.396289 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:11.396289 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:11.396289 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:11.396289 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:11.396289 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:11.396289 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:11.397265 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:11.397265 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:11.397265 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:11.397265 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:11.397265 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:11.397265 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:11.397265 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:11.397265 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:11.397265 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:11.397265 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:11.397265 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:11.397265 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:11.397265 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:11.397265 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:11.397265 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:11.397265 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:11.397265 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:11.403956 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:11.403956 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:11.403956 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:11.403956 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:11.403956 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 281/564] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:11.404257 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b162853d-e7c7-4a4f-b983-789aac6ee3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:11.404350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c36dd0e-9299-4744-b9a3-1e3b4c136c54 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:11.404420 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b162853d-e7c7-4a4f-b983-789aac6ee3d6 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:11.404420 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b162853d-e7c7-4a4f-b983-789aac6ee3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:11.404420 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b162853d-e7c7-4a4f-b983-789aac6ee3d6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:11.404420 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:11.413560 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b162853d-e7c7-4a4f-b983-789aac6ee3d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:11.421577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-b162853d-e7c7-4a4f-b983-789aac6ee3d6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:11.421577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:11.421577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:11.421577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:11.421577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:11.421577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:11.421577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:11.421577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:11.421577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:11.421577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:11.421577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:11.421577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:11.421577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:11.421577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:11.421577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:11.421577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:11.422737 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:11.422737 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:11.422737 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:11.422737 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:11.422737 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:11.422737 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:11.422737 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:11.422737 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:11.422737 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:11.422737 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:11.422737 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:11.422737 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:11.422737 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:11.422737 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:11.422737 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:11.422737 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:11.428875 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:11.428875 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:11.428875 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:11.428875 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:11.428875 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:11.428875 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:11.428875 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:11.428875 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:11.428875 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:11.428875 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:11.428875 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:11.428875 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:11.428875 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 284/565] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 27 21:03:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:11.428875 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b162853d-e7c7-4a4f-b983-789aac6ee3d6 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:11.439378 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:11.528748 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c36dd0e-9299-4744-b9a3-1e3b4c136c54 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:11.537532 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b162853d-e7c7-4a4f-b983-789aac6ee3d6 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:11.562677 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5b58d-a2a2-4a12-b419-af43b7ccac88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:11.563331 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5b58d-a2a2-4a12-b419-af43b7ccac88 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:11.563609 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5b58d-a2a2-4a12-b419-af43b7ccac88 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:11.563874 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5b58d-a2a2-4a12-b419-af43b7ccac88 None admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:11.564629 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a5b58d-a2a2-4a12-b419-af43b7ccac88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:11.564917 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a5b58d-a2a2-4a12-b419-af43b7ccac88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:11.566288 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a5b58d-a2a2-4a12-b419-af43b7ccac88 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:11.571202 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677902ad-dfe0-43b3-a20a-12b1f0ab9ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:11.571998 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677902ad-dfe0-43b3-a20a-12b1f0ab9ec4 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:11.572278 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677902ad-dfe0-43b3-a20a-12b1f0ab9ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:11.572550 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677902ad-dfe0-43b3-a20a-12b1f0ab9ec4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:11.578417 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 282/566] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:11.579033 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5b58d-a2a2-4a12-b419-af43b7ccac88 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:11.587612 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677902ad-dfe0-43b3-a20a-12b1f0ab9ec4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:11.595445 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-677902ad-dfe0-43b3-a20a-12b1f0ab9ec4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:11.595445 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:11.595445 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:11.595445 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:11.595445 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:11.595445 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:11.595445 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:11.595445 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:11.595445 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:11.595445 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:11.595445 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:11.595445 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:11.595445 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:11.595445 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:11.595445 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:11.595445 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:11.596401 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:11.596401 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:11.596401 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:11.596401 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:11.596401 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:11.596401 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:11.596401 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:11.596401 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:11.596401 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:11.596401 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:11.596401 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:11.596401 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:11.596401 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:11.596401 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:11.596401 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:11.596401 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:11.597255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:11.597255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:11.597255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:11.597255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:11.597255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:11.597255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:11.597255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:11.597255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:11.597255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:11.597255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:11.597255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:11.597255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:11.597255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 285/567] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 27 21:03:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:11.598092 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677902ad-dfe0-43b3-a20a-12b1f0ab9ec4 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:11.656127 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:11.670209 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:11.692735 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5b58d-a2a2-4a12-b419-af43b7ccac88 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:11.704447 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677902ad-dfe0-43b3-a20a-12b1f0ab9ec4 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:11.731889 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283246cf-7f8e-466b-b705-8a39ffff40f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:11.732568 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283246cf-7f8e-466b-b705-8a39ffff40f3 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:11.732849 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283246cf-7f8e-466b-b705-8a39ffff40f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:11.733159 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283246cf-7f8e-466b-b705-8a39ffff40f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:11.733899 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283246cf-7f8e-466b-b705-8a39ffff40f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:11.734208 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283246cf-7f8e-466b-b705-8a39ffff40f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:11.735598 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283246cf-7f8e-466b-b705-8a39ffff40f3 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:11.746718 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c6963f-cb5f-49fc-bded-ec9cc559c8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:11.747389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6963f-cb5f-49fc-bded-ec9cc559c8b4 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:11.747668 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6963f-cb5f-49fc-bded-ec9cc559c8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:11.747947 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6963f-cb5f-49fc-bded-ec9cc559c8b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:11.748711 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c6963f-cb5f-49fc-bded-ec9cc559c8b4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:11.748800 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 283/568] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:11.749054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c6963f-cb5f-49fc-bded-ec9cc559c8b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:11.750143 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599620b-d74f-43a1-bf6a-2d4ce6945e3b None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:11.750425 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599620b-d74f-43a1-bf6a-2d4ce6945e3b None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:11.750517 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c6963f-cb5f-49fc-bded-ec9cc559c8b4 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:11.750712 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599620b-d74f-43a1-bf6a-2d4ce6945e3b None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:11.751895 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 284/569] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:11.752606 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3599620b-d74f-43a1-bf6a-2d4ce6945e3b None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:11.764099 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 286/570] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 27 21:03:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:11.764980 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c6963f-cb5f-49fc-bded-ec9cc559c8b4 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:11.881574 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c6963f-cb5f-49fc-bded-ec9cc559c8b4 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:11.883890 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3599620b-d74f-43a1-bf6a-2d4ce6945e3b None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:11.897580 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:11.915682 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f214fb22-ca7c-43a9-990b-f45092326dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:11.916458 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f214fb22-ca7c-43a9-990b-f45092326dc7 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:11.916738 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f214fb22-ca7c-43a9-990b-f45092326dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:11.917071 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f214fb22-ca7c-43a9-990b-f45092326dc7 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:11.917828 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f214fb22-ca7c-43a9-990b-f45092326dc7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:11.918137 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f214fb22-ca7c-43a9-990b-f45092326dc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:11.919070 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcb57ba-eaf6-4c66-8293-f35f161920ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:11.919526 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f214fb22-ca7c-43a9-990b-f45092326dc7 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:11.919875 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb57ba-eaf6-4c66-8293-f35f161920ad None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:11.920152 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb57ba-eaf6-4c66-8293-f35f161920ad None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:11.920421 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcb57ba-eaf6-4c66-8293-f35f161920ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:11.927630 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 287/571] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 27 21:03:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:11.927766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcb57ba-eaf6-4c66-8293-f35f161920ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:11.928422 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f214fb22-ca7c-43a9-990b-f45092326dc7 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:11.929208 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcb57ba-eaf6-4c66-8293-f35f161920ad None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:11.946231 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-0dcb57ba-eaf6-4c66-8293-f35f161920ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:11.946231 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:11.946231 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:11.946231 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:11.946231 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:11.946231 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:11.946231 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:11.946231 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:11.946231 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:11.946231 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:11.946231 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:11.946231 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:11.946231 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:11.946231 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:11.946231 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:11.947195 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:11.947195 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:11.947195 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:11.947195 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:11.947195 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:11.947195 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:11.947195 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:11.947195 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:11.947195 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:11.947195 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:11.947195 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:11.947195 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:11.947195 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:11.947195 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:11.947195 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:11.947195 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:11.947195 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:11.948101 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:11.948101 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:11.948101 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:11.948101 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:11.948101 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:11.948101 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:11.948101 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:11.948101 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:11.948101 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 285/572] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:11.948998 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cecf527-9c8b-4921-bfff-5a2ebc12d087 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:11.949256 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cecf527-9c8b-4921-bfff-5a2ebc12d087 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:11.949798 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cecf527-9c8b-4921-bfff-5a2ebc12d087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:11.955504 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:11.968691 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-1cecf527-9c8b-4921-bfff-5a2ebc12d087 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:11.969939 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-1cecf527-9c8b-4921-bfff-5a2ebc12d087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:12.036700 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f214fb22-ca7c-43a9-990b-f45092326dc7 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:12.047493 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 286/573] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:12.049833 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cecf527-9c8b-4921-bfff-5a2ebc12d087 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:12.074124 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c4aa8-40f8-4e72-a01a-3cfe71cdd9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:12.075004 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c4aa8-40f8-4e72-a01a-3cfe71cdd9fc None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:12.075291 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c4aa8-40f8-4e72-a01a-3cfe71cdd9fc None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:12.075569 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c4aa8-40f8-4e72-a01a-3cfe71cdd9fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:12.082477 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c4aa8-40f8-4e72-a01a-3cfe71cdd9fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:12.083937 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c4aa8-40f8-4e72-a01a-3cfe71cdd9fc None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:12.098510 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-758c4aa8-40f8-4e72-a01a-3cfe71cdd9fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:12.098510 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:12.098510 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:12.098510 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:12.098510 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:12.098510 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:12.098510 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:12.098510 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:12.098510 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:12.098510 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:12.098510 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:12.098510 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:12.098510 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:12.098510 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:12.098510 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:12.099473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:12.099473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:12.099473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:12.099473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:12.099473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:12.099473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:12.099473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:12.099473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:12.099473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:12.099473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:12.099473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:12.099473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:12.099473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:12.099473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:12.099473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:12.099473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:12.099473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:12.100373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:12.100373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:12.100373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:12.100373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:12.100373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:12.100373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:12.100373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:12.100373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:12.100373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 288/574] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:12.100893 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c4aa8-40f8-4e72-a01a-3cfe71cdd9fc None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:12.154534 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:12.160660 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cecf527-9c8b-4921-bfff-5a2ebc12d087 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:12.204132 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:12.211136 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85364c37-9d7e-40c9-a254-66be81384c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:12.211951 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85364c37-9d7e-40c9-a254-66be81384c4c None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:12.212238 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85364c37-9d7e-40c9-a254-66be81384c4c None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:12.212508 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85364c37-9d7e-40c9-a254-66be81384c4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:12.217664 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c4aa8-40f8-4e72-a01a-3cfe71cdd9fc None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:12.223780 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85364c37-9d7e-40c9-a254-66be81384c4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:12.225257 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85364c37-9d7e-40c9-a254-66be81384c4c None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:12.244596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-85364c37-9d7e-40c9-a254-66be81384c4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:12.244596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:12.244596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:12.244596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:12.244596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:12.244596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:12.244596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:12.244596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:12.244596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:12.244596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:12.244596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:12.244596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:12.244596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:12.244596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:12.244596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:12.245889 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:12.245889 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:12.245889 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:12.245889 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:12.245889 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:12.245889 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:12.245889 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:12.245889 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:12.245889 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:12.245889 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:12.245889 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:12.245889 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:12.245889 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:12.245889 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:12.245889 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:12.245889 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:12.245889 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:12.247034 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:12.247034 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:12.247034 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:12.247034 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:12.247034 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 287/575] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:12.247034 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85364c37-9d7e-40c9-a254-66be81384c4c None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:12.255405 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff743c69-e464-44e4-88d8-acf5f180c3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:12.256212 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff743c69-e464-44e4-88d8-acf5f180c3c9 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:12.256490 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff743c69-e464-44e4-88d8-acf5f180c3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:12.256764 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff743c69-e464-44e4-88d8-acf5f180c3c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:12.267720 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff743c69-e464-44e4-88d8-acf5f180c3c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:12.269737 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff743c69-e464-44e4-88d8-acf5f180c3c9 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:12.287720 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-ff743c69-e464-44e4-88d8-acf5f180c3c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:12.287720 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:12.287720 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:12.287720 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:12.287720 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:12.287720 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:12.287720 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:12.287720 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:12.287720 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:12.287720 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:12.287720 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:12.287720 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:12.287720 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:12.287720 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:12.287720 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:12.288831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:12.288831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:12.288831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:12.288831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:12.288831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:12.288831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:12.288831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:12.288831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:12.288831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:12.288831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:12.288831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:12.288831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:12.288831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:12.288831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:12.288831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:12.288831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:12.288831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:12.289769 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:12.289769 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:12.289769 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:12.289769 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:12.293444 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 289/576] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:12.296425 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff743c69-e464-44e4-88d8-acf5f180c3c9 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:12.360201 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85364c37-9d7e-40c9-a254-66be81384c4c None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:12.396945 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79bf2b3-056b-4b4e-8160-475d92aa3fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:12.397628 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79bf2b3-056b-4b4e-8160-475d92aa3fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:12.397906 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79bf2b3-056b-4b4e-8160-475d92aa3fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:12.398179 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79bf2b3-056b-4b4e-8160-475d92aa3fe0 None admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:12.398986 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79bf2b3-056b-4b4e-8160-475d92aa3fe0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:12.399289 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79bf2b3-056b-4b4e-8160-475d92aa3fe0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:12.400725 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79bf2b3-056b-4b4e-8160-475d92aa3fe0 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:12.401920 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:12.413416 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff743c69-e464-44e4-88d8-acf5f180c3c9 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:12.413754 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 288/577] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:12.414580 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79bf2b3-056b-4b4e-8160-475d92aa3fe0 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:12.450457 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d177ac-ba57-4404-976d-81ec2ac6c347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:12.451266 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d177ac-ba57-4404-976d-81ec2ac6c347 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:12.451548 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d177ac-ba57-4404-976d-81ec2ac6c347 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:12.451833 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d177ac-ba57-4404-976d-81ec2ac6c347 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:12.460788 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:12.462285 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d177ac-ba57-4404-976d-81ec2ac6c347 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:12.470411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-20d177ac-ba57-4404-976d-81ec2ac6c347 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:12.470411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:12.470411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:12.470411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:12.470411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:12.470411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:12.470411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:12.470411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:12.470411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:12.470411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:12.470411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:12.470411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:12.470411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:12.470411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:12.470411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:12.470411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:12.471559 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:12.471559 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:12.471559 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:12.471559 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:12.471559 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:12.471559 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:12.471559 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:12.471559 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:12.471559 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:12.471559 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:12.471559 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:12.471559 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:12.471559 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:12.471559 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:12.471559 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:12.471559 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:12.472474 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:12.472474 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:12.472474 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:12.472474 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:12.472474 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:12.472474 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:12.472474 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:12.472474 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:12.472474 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:12.472474 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:12.472474 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:12.472474 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:12.472474 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 290/578] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 27 21:03:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:12.472474 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d177ac-ba57-4404-976d-81ec2ac6c347 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:12.520977 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79bf2b3-056b-4b4e-8160-475d92aa3fe0 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:12.554360 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8c402-7c72-465d-a857-024e1805d4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:12.555023 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8c402-7c72-465d-a857-024e1805d4a6 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:12.555300 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8c402-7c72-465d-a857-024e1805d4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:12.555573 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8c402-7c72-465d-a857-024e1805d4a6 None admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:12.556356 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8c402-7c72-465d-a857-024e1805d4a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:12.556659 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8c402-7c72-465d-a857-024e1805d4a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:12.558319 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8c402-7c72-465d-a857-024e1805d4a6 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:12.570632 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 289/579] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:12.571449 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8c402-7c72-465d-a857-024e1805d4a6 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:12.573128 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d177ac-ba57-4404-976d-81ec2ac6c347 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:12.607553 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75404f8f-85ab-4698-9fea-46bf746e6f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:12.608216 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75404f8f-85ab-4698-9fea-46bf746e6f4d None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:12.608499 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75404f8f-85ab-4698-9fea-46bf746e6f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:12.608774 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75404f8f-85ab-4698-9fea-46bf746e6f4d None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:12.609537 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75404f8f-85ab-4698-9fea-46bf746e6f4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:12.609847 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75404f8f-85ab-4698-9fea-46bf746e6f4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:12.611228 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75404f8f-85ab-4698-9fea-46bf746e6f4d None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:12.626078 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 291/580] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:12.626078 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75404f8f-85ab-4698-9fea-46bf746e6f4d None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:12.638596 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:12.681676 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8c402-7c72-465d-a857-024e1805d4a6 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:12.684485 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:12.721791 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0dbabe-6c3f-46d8-8fa7-0dcd77217a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:12.722436 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0dbabe-6c3f-46d8-8fa7-0dcd77217a92 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:12.722746 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0dbabe-6c3f-46d8-8fa7-0dcd77217a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:12.723022 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0dbabe-6c3f-46d8-8fa7-0dcd77217a92 None admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:12.723856 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0dbabe-6c3f-46d8-8fa7-0dcd77217a92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:12.724196 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0dbabe-6c3f-46d8-8fa7-0dcd77217a92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:12.725711 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0dbabe-6c3f-46d8-8fa7-0dcd77217a92 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:12.737487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75404f8f-85ab-4698-9fea-46bf746e6f4d None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:12.738853 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 290/581] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:12.739710 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0dbabe-6c3f-46d8-8fa7-0dcd77217a92 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:12.779823 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bfd9ab-32bb-4c96-8367-13dc5d94323c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:12.780598 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bfd9ab-32bb-4c96-8367-13dc5d94323c None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:12.780953 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bfd9ab-32bb-4c96-8367-13dc5d94323c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:12.781261 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bfd9ab-32bb-4c96-8367-13dc5d94323c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:12.782215 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bfd9ab-32bb-4c96-8367-13dc5d94323c None admin] RBAC: Adding query filter params (role.id=709aef6b85524eb2a8e9986a2bd9525b, scope.project.id=334438c929c44e489e9a3b93e6d37231, user.id=8bed6b231ee647d8a350da055d8e59d7) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:12.782592 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bfd9ab-32bb-4c96-8367-13dc5d94323c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:12.784288 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bfd9ab-32bb-4c96-8367-13dc5d94323c None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:12.790879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 292/582] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:03:12 2022] GET /identity/v3/role_assignments?user.id=8bed6b231ee647d8a350da055d8e59d7&scope.project.id=334438c929c44e489e9a3b93e6d37231&role.id=709aef6b85524eb2a8e9986a2bd9525b => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:12.791693 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bfd9ab-32bb-4c96-8367-13dc5d94323c None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:12.874487 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0dbabe-6c3f-46d8-8fa7-0dcd77217a92 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:12.889838 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:12.918756 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bfd9ab-32bb-4c96-8367-13dc5d94323c None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:12.927139 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd82c0-d5b1-4355-b9c3-b9f931a32a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:12.927869 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd82c0-d5b1-4355-b9c3-b9f931a32a6b None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:12.928116 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd82c0-d5b1-4355-b9c3-b9f931a32a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:12.928392 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd82c0-d5b1-4355-b9c3-b9f931a32a6b None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:12.929233 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bd82c0-d5b1-4355-b9c3-b9f931a32a6b None admin] RBAC: Adding query filter params (name=service) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:12.929545 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bd82c0-d5b1-4355-b9c3-b9f931a32a6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:12.930998 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bd82c0-d5b1-4355-b9c3-b9f931a32a6b None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:12.946781 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 291/583] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 27 21:03:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:12.946781 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd82c0-d5b1-4355-b9c3-b9f931a32a6b None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:12.972248 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d466e9-9aae-432c-8e36-7b3faf8ee9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:12.973297 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d466e9-9aae-432c-8e36-7b3faf8ee9be None admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:12.973649 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d466e9-9aae-432c-8e36-7b3faf8ee9be None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:12.973935 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d466e9-9aae-432c-8e36-7b3faf8ee9be None admin] PATH_INFO: `/v3/projects/334438c929c44e489e9a3b93e6d37231/users/1366958034d540b4ba2feb421a6afabc/roles/709aef6b85524eb2a8e9986a2bd9525b` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:12.988884 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:12.992438 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d466e9-9aae-432c-8e36-7b3faf8ee9be None admin] RBAC: Authorizing `identity:create_grant(project_id=334438c929c44e489e9a3b93e6d37231, user_id=1366958034d540b4ba2feb421a6afabc, role_id=709aef6b85524eb2a8e9986a2bd9525b)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:12.993952 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d466e9-9aae-432c-8e36-7b3faf8ee9be None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:13.017032 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 293/584] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 27 21:03:12 2022] PUT /identity/v3/projects/334438c929c44e489e9a3b93e6d37231/users/1366958034d540b4ba2feb421a6afabc/roles/709aef6b85524eb2a8e9986a2bd9525b => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:03:13.019078 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d466e9-9aae-432c-8e36-7b3faf8ee9be None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:13.106990 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd82c0-d5b1-4355-b9c3-b9f931a32a6b None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:13.146379 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d466e9-9aae-432c-8e36-7b3faf8ee9be None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:13.152877 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af19eb7-4cbb-4318-b468-0330ac45b1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:13.153747 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af19eb7-4cbb-4318-b468-0330ac45b1ee None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:13.154031 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af19eb7-4cbb-4318-b468-0330ac45b1ee None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:13.154306 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af19eb7-4cbb-4318-b468-0330ac45b1ee None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:13.163501 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af19eb7-4cbb-4318-b468-0330ac45b1ee None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:13.165816 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af19eb7-4cbb-4318-b468-0330ac45b1ee None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:13.180518 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:13.185331 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-9af19eb7-4cbb-4318-b468-0330ac45b1ee None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 21:03:13.185331 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:13.185331 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:13.185331 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:13.185331 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:13.185331 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:13.185331 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:13.185331 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:13.185331 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:13.185331 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:13.185331 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:13.185331 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:13.185331 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:13.185331 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:13.185331 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:13.185331 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:13.186366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:13.186366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:13.186366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:13.186366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:13.186366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:13.186366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:13.186366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:13.186366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:13.186366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:13.186366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:13.186366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:13.186366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:13.186366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:13.186366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:13.186366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:13.186366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:13.186366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:13.186366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:13.187739 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:13.187739 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:13.187739 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:13.187739 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:13.187739 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:13.187739 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:13.187739 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:13.187739 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:13.187739 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 21:03:13.187739 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:13.187739 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 292/585] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:03:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:13.187739 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af19eb7-4cbb-4318-b468-0330ac45b1ee None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:13.201915 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb23f6d0-cb74-4329-8885-b3d440667d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:13.203328 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23f6d0-cb74-4329-8885-b3d440667d79 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:13.203823 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23f6d0-cb74-4329-8885-b3d440667d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:13.204642 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23f6d0-cb74-4329-8885-b3d440667d79 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:13.215577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb23f6d0-cb74-4329-8885-b3d440667d79 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:13.217469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb23f6d0-cb74-4329-8885-b3d440667d79 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:13.241743 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-fb23f6d0-cb74-4329-8885-b3d440667d79 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 27 21:03:13.241743 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:13.241743 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:13.241743 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:13.241743 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:13.241743 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:13.241743 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:13.241743 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:13.241743 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:13.241743 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:13.241743 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:13.241743 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:13.241743 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:13.241743 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:13.241743 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:13.241743 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:13.242734 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:13.242734 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:13.242734 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:13.242734 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:13.242734 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:13.242734 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:13.242734 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:13.242734 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:13.242734 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:13.242734 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:13.242734 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:13.242734 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:13.242734 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:13.242734 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:13.242734 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:13.242734 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:13.242734 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:13.242734 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:13.243672 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:13.243672 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:13.243672 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:13.243672 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:13.243672 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:13.243672 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:13.243672 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:13.243672 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:13.243672 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 27 21:03:13.243672 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:13.246703 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 294/586] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:03:13 2022] GET /identity/v3/users/glance => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:13.247836 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb23f6d0-cb74-4329-8885-b3d440667d79 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:13.305662 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:13.325469 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af19eb7-4cbb-4318-b468-0330ac45b1ee None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:13.366060 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b7022-ac60-4187-890e-5d3d5d36ff79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:13.366877 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b7022-ac60-4187-890e-5d3d5d36ff79 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:13.367159 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b7022-ac60-4187-890e-5d3d5d36ff79 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:13.367433 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b7022-ac60-4187-890e-5d3d5d36ff79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:13.370472 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb23f6d0-cb74-4329-8885-b3d440667d79 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:13.379297 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346b7022-ac60-4187-890e-5d3d5d36ff79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:13.380781 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346b7022-ac60-4187-890e-5d3d5d36ff79 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:13.398549 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-346b7022-ac60-4187-890e-5d3d5d36ff79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:13.398549 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:13.398549 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:13.398549 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:13.398549 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:13.398549 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:13.398549 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:13.398549 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:13.398549 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:13.398549 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:13.398549 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:13.398549 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:13.398549 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:13.398549 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:13.398549 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:13.399507 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:13.399507 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:13.399507 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:13.399507 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:13.399507 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:13.399507 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:13.399507 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:13.399507 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:13.399507 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:13.399507 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:13.399507 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:13.399507 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:13.399507 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:13.399507 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:13.399507 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:13.399507 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:13.399507 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:13.400854 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:13.400854 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:13.400854 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:13.400854 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:13.400854 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 293/587] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:13.401178 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b7022-ac60-4187-890e-5d3d5d36ff79 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:13.409804 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25505cf0-b846-4c24-b035-c21e826a03f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:13.410486 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25505cf0-b846-4c24-b035-c21e826a03f4 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:13.410767 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25505cf0-b846-4c24-b035-c21e826a03f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:13.411040 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25505cf0-b846-4c24-b035-c21e826a03f4 None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:13.411895 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25505cf0-b846-4c24-b035-c21e826a03f4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:13.412204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25505cf0-b846-4c24-b035-c21e826a03f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:13.413690 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25505cf0-b846-4c24-b035-c21e826a03f4 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:13.437903 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 295/588] 10.222.0.13 () {56 vars in 1137 bytes} [Mon Jun 27 21:03:13 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:13.439051 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25505cf0-b846-4c24-b035-c21e826a03f4 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:13.487745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:13.510928 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b7022-ac60-4187-890e-5d3d5d36ff79 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:13.554893 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25505cf0-b846-4c24-b035-c21e826a03f4 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:13.559493 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba4d2b-0b9e-4c1a-87cf-a6c04b076f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:13.560456 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba4d2b-0b9e-4c1a-87cf-a6c04b076f1a None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:13.561016 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba4d2b-0b9e-4c1a-87cf-a6c04b076f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:13.561508 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba4d2b-0b9e-4c1a-87cf-a6c04b076f1a None admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:13.562655 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bba4d2b-0b9e-4c1a-87cf-a6c04b076f1a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:13.563679 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bba4d2b-0b9e-4c1a-87cf-a6c04b076f1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:13.565911 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bba4d2b-0b9e-4c1a-87cf-a6c04b076f1a None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:13.578810 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:13.596901 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 294/589] 10.222.0.13 () {56 vars in 1137 bytes} [Mon Jun 27 21:03:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:13.597615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba4d2b-0b9e-4c1a-87cf-a6c04b076f1a None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:13.601042 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61745ff0-abbd-47f9-962c-718a0e734c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:13.602137 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61745ff0-abbd-47f9-962c-718a0e734c83 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:13.602575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61745ff0-abbd-47f9-962c-718a0e734c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:13.602772 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61745ff0-abbd-47f9-962c-718a0e734c83 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:13.603516 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61745ff0-abbd-47f9-962c-718a0e734c83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:13.603823 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61745ff0-abbd-47f9-962c-718a0e734c83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:13.605507 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61745ff0-abbd-47f9-962c-718a0e734c83 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:13.625001 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 296/590] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:13.625001 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61745ff0-abbd-47f9-962c-718a0e734c83 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:13.737394 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba4d2b-0b9e-4c1a-87cf-a6c04b076f1a None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:13.779926 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d67745-e808-4456-964e-b3b62e4ac8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:13.781252 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d67745-e808-4456-964e-b3b62e4ac8b3 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:13.781704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d67745-e808-4456-964e-b3b62e4ac8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:13.782312 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d67745-e808-4456-964e-b3b62e4ac8b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:13.785896 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61745ff0-abbd-47f9-962c-718a0e734c83 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:13.787743 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:13.798828 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d67745-e808-4456-964e-b3b62e4ac8b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:13.801021 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d67745-e808-4456-964e-b3b62e4ac8b3 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:13.820865 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-41d67745-e808-4456-964e-b3b62e4ac8b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:13.820865 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:13.820865 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:13.820865 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:13.820865 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:13.820865 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:13.820865 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:13.820865 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:13.820865 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:13.820865 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:13.820865 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:13.820865 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:13.820865 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:13.820865 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:13.820865 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:13.822042 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:13.822042 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:13.822042 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:13.822042 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:13.822042 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:13.822042 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:13.822042 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:13.822042 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:13.822042 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:13.822042 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:13.822042 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:13.822042 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:13.822042 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:13.822042 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:13.822042 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:13.822042 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:13.822042 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:13.823123 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:13.823123 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:13.823123 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:13.823123 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:13.830295 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 295/591] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:13.830295 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d67745-e808-4456-964e-b3b62e4ac8b3 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:13.836261 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b411c-cdbc-4ac0-a2fb-e3af44644e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:13.837328 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b411c-cdbc-4ac0-a2fb-e3af44644e0c None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:13.837769 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b411c-cdbc-4ac0-a2fb-e3af44644e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:13.838187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b411c-cdbc-4ac0-a2fb-e3af44644e0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:13.853588 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6b411c-cdbc-4ac0-a2fb-e3af44644e0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:13.856046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6b411c-cdbc-4ac0-a2fb-e3af44644e0c None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:13.878943 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-4c6b411c-cdbc-4ac0-a2fb-e3af44644e0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:13.878943 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:13.878943 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:13.878943 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:13.878943 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:13.878943 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:13.878943 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:13.878943 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:13.878943 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:13.878943 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:13.878943 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:13.878943 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:13.878943 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:13.878943 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:13.878943 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:13.880016 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:13.880016 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:13.880016 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:13.880016 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:13.880016 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:13.880016 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:13.880016 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:13.880016 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:13.880016 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:13.880016 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:13.880016 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:13.880016 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:13.880016 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:13.880016 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:13.880016 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:13.880016 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:13.880016 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:13.880912 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:13.880912 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:13.880912 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:13.880912 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:13.885848 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 297/592] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:13.886746 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b411c-cdbc-4ac0-a2fb-e3af44644e0c None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:13.890372 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:13.951861 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d67745-e808-4456-964e-b3b62e4ac8b3 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:13.991219 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638db6fb-f41b-4995-a44e-f85b3d7d5c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:13.992442 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638db6fb-f41b-4995-a44e-f85b3d7d5c45 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:13.992949 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638db6fb-f41b-4995-a44e-f85b3d7d5c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:13.993402 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638db6fb-f41b-4995-a44e-f85b3d7d5c45 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:13.994968 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b411c-cdbc-4ac0-a2fb-e3af44644e0c None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:14.001573 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638db6fb-f41b-4995-a44e-f85b3d7d5c45 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:14.003352 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638db6fb-f41b-4995-a44e-f85b3d7d5c45 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:14.020593 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-638db6fb-f41b-4995-a44e-f85b3d7d5c45 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 27 21:03:14.020593 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:14.020593 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:14.020593 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:14.020593 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:14.020593 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:14.020593 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:14.020593 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:14.020593 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:14.020593 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:14.020593 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:14.020593 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:14.020593 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:14.020593 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:14.020593 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:14.020593 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:14.021561 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:14.021561 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:14.021561 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:14.021561 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:14.021561 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:14.021561 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:14.021561 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:14.021561 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:14.021561 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:14.021561 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:14.021561 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:14.021561 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:14.021561 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:14.021561 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:14.021561 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:14.021561 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:14.021561 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:14.021561 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:14.022500 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:14.022500 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:14.022500 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:14.022500 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:14.022500 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:14.022500 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:14.022500 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:14.022500 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:14.022500 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 27 21:03:14.022500 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:14.030267 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 296/593] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 27 21:03:13 2022] GET /identity/v3/users/neutron => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:14.030267 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638db6fb-f41b-4995-a44e-f85b3d7d5c45 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:14.034146 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b0ef8-8768-4b19-ac29-46d20a0440f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:14.034830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b0ef8-8768-4b19-ac29-46d20a0440f0 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:14.035106 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b0ef8-8768-4b19-ac29-46d20a0440f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:14.036181 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b0ef8-8768-4b19-ac29-46d20a0440f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:14.036181 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b0ef8-8768-4b19-ac29-46d20a0440f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:14.036437 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b0ef8-8768-4b19-ac29-46d20a0440f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:14.037834 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b0ef8-8768-4b19-ac29-46d20a0440f0 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:14.057294 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 298/594] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:14.058888 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b0ef8-8768-4b19-ac29-46d20a0440f0 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:14.082454 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:14.170102 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638db6fb-f41b-4995-a44e-f85b3d7d5c45 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:14.180889 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b0ef8-8768-4b19-ac29-46d20a0440f0 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:14.189325 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:14.214597 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1125140-9589-4e8d-b64d-232591c8cafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:14.215550 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1125140-9589-4e8d-b64d-232591c8cafa None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:14.215964 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1125140-9589-4e8d-b64d-232591c8cafa None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:14.216254 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1125140-9589-4e8d-b64d-232591c8cafa None admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:14.217066 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1125140-9589-4e8d-b64d-232591c8cafa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:14.217384 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1125140-9589-4e8d-b64d-232591c8cafa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:14.218641 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb98116b-cb0d-48fa-bb0c-d51aef90d048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:14.219380 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1125140-9589-4e8d-b64d-232591c8cafa None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:14.219494 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98116b-cb0d-48fa-bb0c-d51aef90d048 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:14.219779 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98116b-cb0d-48fa-bb0c-d51aef90d048 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:14.220116 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98116b-cb0d-48fa-bb0c-d51aef90d048 None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:14.221051 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb98116b-cb0d-48fa-bb0c-d51aef90d048 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:14.221426 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb98116b-cb0d-48fa-bb0c-d51aef90d048 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:14.222922 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb98116b-cb0d-48fa-bb0c-d51aef90d048 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:14.235494 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 297/595] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:14.236498 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1125140-9589-4e8d-b64d-232591c8cafa None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:14.242811 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 299/596] 10.222.0.13 () {56 vars in 1139 bytes} [Mon Jun 27 21:03:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:14.243831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb98116b-cb0d-48fa-bb0c-d51aef90d048 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:14.342489 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:14.362708 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1125140-9589-4e8d-b64d-232591c8cafa None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:14.387260 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb98116b-cb0d-48fa-bb0c-d51aef90d048 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:14.399310 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9b06f2-1bea-447d-9b19-84de910b79a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:14.400303 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9b06f2-1bea-447d-9b19-84de910b79a9 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:14.400677 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9b06f2-1bea-447d-9b19-84de910b79a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:14.401007 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9b06f2-1bea-447d-9b19-84de910b79a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:14.410275 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9b06f2-1bea-447d-9b19-84de910b79a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:14.411832 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9b06f2-1bea-447d-9b19-84de910b79a9 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:14.427384 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277ca3ce-c281-4d1a-8e64-1aa52b8c67ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:14.427918 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-3b9b06f2-1bea-447d-9b19-84de910b79a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:14.427918 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:14.427918 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:14.427918 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:14.427918 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:14.427918 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:14.427918 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:14.427918 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:14.427918 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:14.427918 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:14.427918 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:14.427918 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:14.427918 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:14.427918 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:14.427918 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:14.429007 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:14.429007 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:14.429007 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:14.429007 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:14.429007 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:14.429007 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:14.429007 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:14.429007 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:14.429007 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:14.429007 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:14.429007 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:14.429007 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:14.429007 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:14.429007 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:14.429007 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:14.429007 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:14.429007 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:14.429938 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:14.429938 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:14.429938 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:14.429938 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:14.429938 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:14.429938 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:14.429938 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:14.429938 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:14.430537 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 298/597] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:14.431444 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9b06f2-1bea-447d-9b19-84de910b79a9 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:14.431780 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ca3ce-c281-4d1a-8e64-1aa52b8c67ed None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:14.432226 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ca3ce-c281-4d1a-8e64-1aa52b8c67ed None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:14.433226 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ca3ce-c281-4d1a-8e64-1aa52b8c67ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:14.441465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277ca3ce-c281-4d1a-8e64-1aa52b8c67ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:14.443176 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277ca3ce-c281-4d1a-8e64-1aa52b8c67ed None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:14.457849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-277ca3ce-c281-4d1a-8e64-1aa52b8c67ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:14.457849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:14.457849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:14.457849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:14.457849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:14.457849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:14.457849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:14.457849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:14.457849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:14.457849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:14.457849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:14.457849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:14.457849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:14.457849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:14.457849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:14.458928 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:14.458928 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:14.458928 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:14.458928 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:14.458928 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:14.458928 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:14.458928 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:14.458928 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:14.458928 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:14.458928 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:14.458928 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:14.458928 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:14.458928 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:14.458928 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:14.458928 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:14.458928 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:14.458928 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:14.459860 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:14.459860 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:14.459860 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:14.459860 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:14.459860 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:14.459860 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:14.459860 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:14.459860 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:14.462555 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 300/598] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:14.463761 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277ca3ce-c281-4d1a-8e64-1aa52b8c67ed None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:14.475781 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:14.560542 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9b06f2-1bea-447d-9b19-84de910b79a9 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:14.571678 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277ca3ce-c281-4d1a-8e64-1aa52b8c67ed None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:14.598690 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eec565-8515-4140-91b4-d4759633adb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:14.599610 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eec565-8515-4140-91b4-d4759633adb3 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:14.599804 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eec565-8515-4140-91b4-d4759633adb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:14.600061 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eec565-8515-4140-91b4-d4759633adb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:14.603492 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:14.609250 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c0dc7a-4629-4499-8303-33c65e9500c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:14.610564 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0dc7a-4629-4499-8303-33c65e9500c9 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:14.611052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0dc7a-4629-4499-8303-33c65e9500c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:14.611549 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0dc7a-4629-4499-8303-33c65e9500c9 None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:14.612559 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c0dc7a-4629-4499-8303-33c65e9500c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:14.613115 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c0dc7a-4629-4499-8303-33c65e9500c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:14.613392 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eec565-8515-4140-91b4-d4759633adb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:14.614862 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eec565-8515-4140-91b4-d4759633adb3 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:14.615204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c0dc7a-4629-4499-8303-33c65e9500c9 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:14.628408 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 301/599] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:14.629679 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c0dc7a-4629-4499-8303-33c65e9500c9 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:14.634687 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-d2eec565-8515-4140-91b4-d4759633adb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:14.634687 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:14.634687 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:14.634687 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:14.634687 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:14.634687 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:14.634687 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:14.634687 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:14.634687 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:14.634687 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:14.634687 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:14.634687 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:14.634687 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:14.634687 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:14.634687 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:14.635587 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:14.635587 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:14.635587 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:14.635587 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:14.635587 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:14.635587 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:14.635587 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:14.635587 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:14.635587 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:14.635587 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:14.635587 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:14.635587 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:14.635587 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:14.635587 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:14.635587 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:14.635587 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:14.635587 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:14.637236 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:14.637236 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:14.637236 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:14.637236 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:14.637236 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 299/600] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:14.637236 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eec565-8515-4140-91b4-d4759633adb3 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:14.726043 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:14.737038 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c0dc7a-4629-4499-8303-33c65e9500c9 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:14.749148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eec565-8515-4140-91b4-d4759633adb3 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:14.773020 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4090d8e2-cd8c-4766-81a4-bfb29af0cd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:14.773697 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090d8e2-cd8c-4766-81a4-bfb29af0cd13 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:14.773999 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090d8e2-cd8c-4766-81a4-bfb29af0cd13 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:14.774236 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090d8e2-cd8c-4766-81a4-bfb29af0cd13 None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:14.775035 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4090d8e2-cd8c-4766-81a4-bfb29af0cd13 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:14.775342 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4090d8e2-cd8c-4766-81a4-bfb29af0cd13 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:14.776863 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4090d8e2-cd8c-4766-81a4-bfb29af0cd13 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:14.788949 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5a8c2e-1ca6-466f-ba94-63a8aaefaa13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:14.789320 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 302/601] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:14.790227 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4090d8e2-cd8c-4766-81a4-bfb29af0cd13 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:14.790632 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a8c2e-1ca6-466f-ba94-63a8aaefaa13 None admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:14.791011 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a8c2e-1ca6-466f-ba94-63a8aaefaa13 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:14.791378 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a8c2e-1ca6-466f-ba94-63a8aaefaa13 None admin] PATH_INFO: `/v3/projects/334438c929c44e489e9a3b93e6d37231/users/340915e7debe4ce6828bd8a477ed6f16/roles/709aef6b85524eb2a8e9986a2bd9525b` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:14.808224 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5a8c2e-1ca6-466f-ba94-63a8aaefaa13 None admin] RBAC: Authorizing `identity:create_grant(project_id=334438c929c44e489e9a3b93e6d37231, user_id=340915e7debe4ce6828bd8a477ed6f16, role_id=709aef6b85524eb2a8e9986a2bd9525b)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:14.810139 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5a8c2e-1ca6-466f-ba94-63a8aaefaa13 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:14.832584 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 300/602] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 27 21:03:14 2022] PUT /identity/v3/projects/334438c929c44e489e9a3b93e6d37231/users/340915e7debe4ce6828bd8a477ed6f16/roles/709aef6b85524eb2a8e9986a2bd9525b => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:03:14.833600 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5a8c2e-1ca6-466f-ba94-63a8aaefaa13 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:14.861100 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:14.963361 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5a8c2e-1ca6-466f-ba94-63a8aaefaa13 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:14.966471 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4090d8e2-cd8c-4766-81a4-bfb29af0cd13 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:15.002946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:15.018733 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf79524-c573-4e4d-b55c-7d6024d064fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:15.020000 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf79524-c573-4e4d-b55c-7d6024d064fe None admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:15.020510 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf79524-c573-4e4d-b55c-7d6024d064fe None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:15.021015 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf79524-c573-4e4d-b55c-7d6024d064fe None admin] PATH_INFO: `/v3/projects/334438c929c44e489e9a3b93e6d37231/users/26cf962d151b4da7a82c33ef23b342e6/roles/709aef6b85524eb2a8e9986a2bd9525b` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:15.044135 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf79524-c573-4e4d-b55c-7d6024d064fe None admin] RBAC: Authorizing `identity:create_grant(project_id=334438c929c44e489e9a3b93e6d37231, user_id=26cf962d151b4da7a82c33ef23b342e6, role_id=709aef6b85524eb2a8e9986a2bd9525b)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:15.046688 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf79524-c573-4e4d-b55c-7d6024d064fe None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:15.067233 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9f90e-f9ce-4f92-87db-88e3ea3deb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:15.068680 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9f90e-f9ce-4f92-87db-88e3ea3deb3f None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:15.068680 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9f90e-f9ce-4f92-87db-88e3ea3deb3f None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:15.068680 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9f90e-f9ce-4f92-87db-88e3ea3deb3f None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:15.071050 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea9f90e-f9ce-4f92-87db-88e3ea3deb3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:15.071050 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea9f90e-f9ce-4f92-87db-88e3ea3deb3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:15.071655 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea9f90e-f9ce-4f92-87db-88e3ea3deb3f None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:15.075738 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 301/603] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 27 21:03:14 2022] PUT /identity/v3/projects/334438c929c44e489e9a3b93e6d37231/users/26cf962d151b4da7a82c33ef23b342e6/roles/709aef6b85524eb2a8e9986a2bd9525b => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:03:15.088735 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 303/604] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:15.094217 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf79524-c573-4e4d-b55c-7d6024d064fe None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:15.118488 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff84e7-e8f2-4624-911d-a44e68e9f80e None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:15.119221 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff84e7-e8f2-4624-911d-a44e68e9f80e None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:15.119712 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff84e7-e8f2-4624-911d-a44e68e9f80e None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:15.122234 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 304/605] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:03:15.129151 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f514a6-8db7-40c8-9281-964fc7ec5c5f None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:15.129795 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f514a6-8db7-40c8-9281-964fc7ec5c5f None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:15.130286 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f514a6-8db7-40c8-9281-964fc7ec5c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:15.153552 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-22f514a6-8db7-40c8-9281-964fc7ec5c5f None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:15.156023 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-22f514a6-8db7-40c8-9281-964fc7ec5c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:15.244408 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf79524-c573-4e4d-b55c-7d6024d064fe None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:15.249033 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 305/606] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:15.250589 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b4e98-b729-453d-aef8-d1704af6de01 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:15.250946 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b4e98-b729-453d-aef8-d1704af6de01 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:15.251301 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b4e98-b729-453d-aef8-d1704af6de01 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:15.253128 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 306/607] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:15.258501 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816947fd-81d3-48c1-99c3-7b7bd7088377 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:15.258912 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816947fd-81d3-48c1-99c3-7b7bd7088377 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:15.259283 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816947fd-81d3-48c1-99c3-7b7bd7088377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:15.288331 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-816947fd-81d3-48c1-99c3-7b7bd7088377 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:15.289694 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-816947fd-81d3-48c1-99c3-7b7bd7088377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:15.292690 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36abd451-99db-4b7c-9421-82f10e510f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:15.293560 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36abd451-99db-4b7c-9421-82f10e510f5a None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:15.293845 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36abd451-99db-4b7c-9421-82f10e510f5a None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:15.294117 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36abd451-99db-4b7c-9421-82f10e510f5a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:15.303278 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36abd451-99db-4b7c-9421-82f10e510f5a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:15.305319 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36abd451-99db-4b7c-9421-82f10e510f5a None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:15.321725 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-36abd451-99db-4b7c-9421-82f10e510f5a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:15.321725 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:15.321725 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:15.321725 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:15.321725 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:15.321725 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:15.321725 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:15.321725 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:15.321725 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:15.321725 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:15.321725 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:15.321725 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:15.321725 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:15.321725 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:15.321725 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:15.322674 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:15.322674 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:15.322674 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:15.322674 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:15.322674 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:15.322674 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:15.322674 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:15.322674 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:15.322674 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:15.322674 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:15.322674 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:15.322674 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:15.322674 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:15.322674 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:15.322674 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:15.322674 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:15.322674 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:15.323633 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:15.323633 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:15.323633 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:15.323633 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:15.323633 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:15.323633 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:15.323633 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:15.323633 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:15.326557 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 302/608] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:15.327791 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36abd451-99db-4b7c-9421-82f10e510f5a None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:15.329847 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:15.373939 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 307/609] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:03:15.375110 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816947fd-81d3-48c1-99c3-7b7bd7088377 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:15.385392 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:15.465534 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36abd451-99db-4b7c-9421-82f10e510f5a None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:15.487109 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816947fd-81d3-48c1-99c3-7b7bd7088377 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:15.507699 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b9a51-9c2e-4dd2-a052-b953cb29e6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:15.509151 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b9a51-9c2e-4dd2-a052-b953cb29e6c6 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:15.509602 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b9a51-9c2e-4dd2-a052-b953cb29e6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:15.510024 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b9a51-9c2e-4dd2-a052-b953cb29e6c6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:15.522131 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565b9a51-9c2e-4dd2-a052-b953cb29e6c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:15.530822 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4bfc51-5ca9-44ce-b7cb-24692ff395c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:15.531628 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bfc51-5ca9-44ce-b7cb-24692ff395c7 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:15.531857 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-565b9a51-9c2e-4dd2-a052-b953cb29e6c6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:15.531857 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:15.531857 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:15.531857 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:15.531857 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:15.531857 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:15.531857 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:15.531857 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:15.531857 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:15.531857 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:15.531857 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:15.531857 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:15.531857 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:15.531857 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:15.531857 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:15.531857 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:15.532855 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:15.532855 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:15.532855 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:15.532855 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:15.532855 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:15.532855 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:15.532855 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:15.532855 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:15.532855 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:15.532855 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:15.532855 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:15.532855 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:15.532855 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:15.532855 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:15.532855 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:15.532855 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:15.533727 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:15.533727 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:15.533727 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:15.533727 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:15.533727 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:15.533727 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:15.533727 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:15.533727 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:15.533727 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:15.533727 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:15.533727 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:15.533727 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:15.534377 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bfc51-5ca9-44ce-b7cb-24692ff395c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:15.534377 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bfc51-5ca9-44ce-b7cb-24692ff395c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:15.534377 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4bfc51-5ca9-44ce-b7cb-24692ff395c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:15.534377 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4bfc51-5ca9-44ce-b7cb-24692ff395c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:15.535480 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4bfc51-5ca9-44ce-b7cb-24692ff395c7 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:15.537144 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 303/610] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 27 21:03:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:15.538426 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b9a51-9c2e-4dd2-a052-b953cb29e6c6 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:15.550128 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 308/611] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:15.551016 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4bfc51-5ca9-44ce-b7cb-24692ff395c7 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:15.590873 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:15.648355 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b9a51-9c2e-4dd2-a052-b953cb29e6c6 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:15.657050 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:15.681350 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4bfc51-5ca9-44ce-b7cb-24692ff395c7 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:15.691045 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fab0f1-7ed5-4dde-ab4e-9397c00f1d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:15.692111 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fab0f1-7ed5-4dde-ab4e-9397c00f1d51 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:15.692496 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fab0f1-7ed5-4dde-ab4e-9397c00f1d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:15.692871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fab0f1-7ed5-4dde-ab4e-9397c00f1d51 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:15.707291 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fab0f1-7ed5-4dde-ab4e-9397c00f1d51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:15.716388 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-e0fab0f1-7ed5-4dde-ab4e-9397c00f1d51 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:15.716388 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:15.716388 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:15.716388 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:15.716388 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:15.716388 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:15.716388 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:15.716388 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:15.716388 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:15.716388 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:15.716388 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:15.716388 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:15.716388 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:15.716388 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:15.716388 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:15.716388 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:15.717480 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:15.717480 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:15.717480 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:15.717480 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:15.717480 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:15.717480 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:15.717480 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:15.717480 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:15.717480 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:15.717480 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:15.717480 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:15.717480 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:15.717480 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:15.717480 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:15.717480 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:15.717480 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:15.718313 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:15.718313 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:15.718313 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:15.718313 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:15.718313 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:15.718313 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:15.718313 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:15.718313 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:15.718313 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:15.718313 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:15.718313 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:15.718313 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:15.718943 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 304/612] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 27 21:03:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:15.720957 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fab0f1-7ed5-4dde-ab4e-9397c00f1d51 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:15.723705 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a04a1-b438-4459-8613-cf4520b4f005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:15.724582 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a04a1-b438-4459-8613-cf4520b4f005 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:15.724859 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a04a1-b438-4459-8613-cf4520b4f005 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:15.725180 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a04a1-b438-4459-8613-cf4520b4f005 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:15.727215 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971a04a1-b438-4459-8613-cf4520b4f005 None admin] RBAC: Adding query filter params (name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:15.727522 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971a04a1-b438-4459-8613-cf4520b4f005 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:15.728909 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971a04a1-b438-4459-8613-cf4520b4f005 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:15.740528 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 309/613] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 27 21:03:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:15.741400 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a04a1-b438-4459-8613-cf4520b4f005 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:15.858897 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fab0f1-7ed5-4dde-ab4e-9397c00f1d51 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:15.865550 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:15.873916 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a04a1-b438-4459-8613-cf4520b4f005 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:15.902035 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041cddca-c09a-466c-a0aa-d7cd78c4e138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:15.902958 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041cddca-c09a-466c-a0aa-d7cd78c4e138 None admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:15.903313 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041cddca-c09a-466c-a0aa-d7cd78c4e138 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:15.903655 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041cddca-c09a-466c-a0aa-d7cd78c4e138 None admin] PATH_INFO: `/v3/projects/334438c929c44e489e9a3b93e6d37231/users/2a4e105858324391b5142d57e34aff9c/roles/709aef6b85524eb2a8e9986a2bd9525b` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:15.915873 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b4fa3a-f656-41ef-b6b8-2bc38e09672f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:15.916808 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4fa3a-f656-41ef-b6b8-2bc38e09672f None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:15.917337 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4fa3a-f656-41ef-b6b8-2bc38e09672f None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:15.917440 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b4fa3a-f656-41ef-b6b8-2bc38e09672f None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:15.918308 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b4fa3a-f656-41ef-b6b8-2bc38e09672f None admin] RBAC: Adding query filter params (name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:15.918610 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b4fa3a-f656-41ef-b6b8-2bc38e09672f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:15.920724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b4fa3a-f656-41ef-b6b8-2bc38e09672f None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:15.921958 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041cddca-c09a-466c-a0aa-d7cd78c4e138 None admin] RBAC: Authorizing `identity:create_grant(project_id=334438c929c44e489e9a3b93e6d37231, user_id=2a4e105858324391b5142d57e34aff9c, role_id=709aef6b85524eb2a8e9986a2bd9525b)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:15.923737 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041cddca-c09a-466c-a0aa-d7cd78c4e138 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:15.930968 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 310/614] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 27 21:03:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:15.931828 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b4fa3a-f656-41ef-b6b8-2bc38e09672f None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:15.946260 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 305/615] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 27 21:03:15 2022] PUT /identity/v3/projects/334438c929c44e489e9a3b93e6d37231/users/2a4e105858324391b5142d57e34aff9c/roles/709aef6b85524eb2a8e9986a2bd9525b => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:03:15.948005 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041cddca-c09a-466c-a0aa-d7cd78c4e138 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:15.971885 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:16.065301 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b4fa3a-f656-41ef-b6b8-2bc38e09672f None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:16.104734 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bad9fe-2291-45b8-8a0d-0bf820cbd3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:16.105638 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bad9fe-2291-45b8-8a0d-0bf820cbd3cf None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:16.105881 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bad9fe-2291-45b8-8a0d-0bf820cbd3cf None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:16.106151 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bad9fe-2291-45b8-8a0d-0bf820cbd3cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:16.128046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bad9fe-2291-45b8-8a0d-0bf820cbd3cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:16.129606 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bad9fe-2291-45b8-8a0d-0bf820cbd3cf None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:16.162993 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-24bad9fe-2291-45b8-8a0d-0bf820cbd3cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:16.162993 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:16.162993 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:16.162993 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:16.162993 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:16.162993 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:16.162993 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:16.162993 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:16.162993 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:16.162993 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:16.162993 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:16.162993 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:16.162993 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:16.162993 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:16.162993 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:16.163938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:16.163938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:16.163938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:16.163938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:16.163938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:16.163938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:16.163938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:16.163938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:16.163938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:16.163938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:16.163938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:16.163938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:16.163938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:16.163938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:16.163938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:16.163938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:16.163938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:16.166063 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:16.166063 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:16.166063 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:16.166063 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:16.166063 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 311/616] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:16.170154 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bad9fe-2291-45b8-8a0d-0bf820cbd3cf None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:16.177459 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041cddca-c09a-466c-a0aa-d7cd78c4e138 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:16.184589 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:16.221606 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d2cc1b-f8a9-4cf7-9464-a361bafc3eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:16.222867 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d2cc1b-f8a9-4cf7-9464-a361bafc3eef None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:16.223515 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d2cc1b-f8a9-4cf7-9464-a361bafc3eef None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:16.224038 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d2cc1b-f8a9-4cf7-9464-a361bafc3eef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:16.237115 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d2cc1b-f8a9-4cf7-9464-a361bafc3eef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:16.238973 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d2cc1b-f8a9-4cf7-9464-a361bafc3eef None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:16.260492 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-e7d2cc1b-f8a9-4cf7-9464-a361bafc3eef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:16.260492 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:16.260492 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:16.260492 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:16.260492 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:16.260492 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:16.260492 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:16.260492 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:16.260492 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:16.260492 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:16.260492 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:16.260492 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:16.260492 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:16.260492 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:16.260492 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:16.261766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:16.261766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:16.261766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:16.261766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:16.261766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:16.261766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:16.261766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:16.261766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:16.261766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:16.261766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:16.261766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:16.261766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:16.261766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:16.261766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:16.261766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:16.261766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:16.261766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:16.262662 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:16.262662 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:16.262662 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:16.262662 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:16.269140 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 306/617] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:16.271461 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d2cc1b-f8a9-4cf7-9464-a361bafc3eef None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:16.303459 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bad9fe-2291-45b8-8a0d-0bf820cbd3cf None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:16.350491 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c2d72f-40e0-4c6f-aa88-b71314e0a429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:16.352026 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c2d72f-40e0-4c6f-aa88-b71314e0a429 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:16.352381 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c2d72f-40e0-4c6f-aa88-b71314e0a429 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:16.352663 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c2d72f-40e0-4c6f-aa88-b71314e0a429 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:16.353475 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c2d72f-40e0-4c6f-aa88-b71314e0a429 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:16.354052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c2d72f-40e0-4c6f-aa88-b71314e0a429 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:16.356017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c2d72f-40e0-4c6f-aa88-b71314e0a429 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:16.372242 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 312/618] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:16.380326 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c2d72f-40e0-4c6f-aa88-b71314e0a429 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:16.430329 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:16.520229 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d2cc1b-f8a9-4cf7-9464-a361bafc3eef None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:16.523848 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:16.590020 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c2d72f-40e0-4c6f-aa88-b71314e0a429 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:16.602013 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb4670-a7f1-4f2c-a0a2-033503b482f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:16.602715 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb4670-a7f1-4f2c-a0a2-033503b482f2 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:16.603005 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb4670-a7f1-4f2c-a0a2-033503b482f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:16.603281 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb4670-a7f1-4f2c-a0a2-033503b482f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:16.604033 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb4670-a7f1-4f2c-a0a2-033503b482f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:16.604381 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb4670-a7f1-4f2c-a0a2-033503b482f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:16.605815 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb4670-a7f1-4f2c-a0a2-033503b482f2 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:16.622701 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 307/619] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:16.628436 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb4670-a7f1-4f2c-a0a2-033503b482f2 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:16.635568 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7170c756-e48f-4ea6-a2bc-f83eb594b25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:16.636560 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170c756-e48f-4ea6-a2bc-f83eb594b25c None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:16.636836 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170c756-e48f-4ea6-a2bc-f83eb594b25c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:16.637154 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170c756-e48f-4ea6-a2bc-f83eb594b25c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:16.653565 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7170c756-e48f-4ea6-a2bc-f83eb594b25c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:16.655231 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7170c756-e48f-4ea6-a2bc-f83eb594b25c None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:16.682724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-7170c756-e48f-4ea6-a2bc-f83eb594b25c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 27 21:03:16.682724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:16.682724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:16.682724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:16.682724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:16.682724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:16.682724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:16.682724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:16.682724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:16.682724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:16.682724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:16.682724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:16.682724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:16.682724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:16.682724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:16.682724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:16.683975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:16.683975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:16.683975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:16.683975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:16.683975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:16.683975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:16.683975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:16.683975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:16.683975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:16.683975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:16.683975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:16.683975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:16.683975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:16.683975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:16.683975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:16.683975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:16.683975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:16.683975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:16.684975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:16.684975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:16.684975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:16.684975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:16.684975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:16.684975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:16.684975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:16.684975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:16.684975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 27 21:03:16.684975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:16.684975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 313/620] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 27 21:03:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:16.694541 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7170c756-e48f-4ea6-a2bc-f83eb594b25c None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:16.821490 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb4670-a7f1-4f2c-a0a2-033503b482f2 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:16.825109 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7170c756-e48f-4ea6-a2bc-f83eb594b25c None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:16.860592 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:16.866645 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:16.871290 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31a14e5-eb9b-40ba-952e-7a3e4fa25df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:16.872092 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a14e5-eb9b-40ba-952e-7a3e4fa25df9 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:16.872441 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a14e5-eb9b-40ba-952e-7a3e4fa25df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:16.872782 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a14e5-eb9b-40ba-952e-7a3e4fa25df9 None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:16.873747 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31a14e5-eb9b-40ba-952e-7a3e4fa25df9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:16.874124 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31a14e5-eb9b-40ba-952e-7a3e4fa25df9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:16.875803 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31a14e5-eb9b-40ba-952e-7a3e4fa25df9 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:16.893889 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854fda8d-b6b9-4d8c-871e-6a9a8894e9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:16.894710 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854fda8d-b6b9-4d8c-871e-6a9a8894e9ee None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:16.894990 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854fda8d-b6b9-4d8c-871e-6a9a8894e9ee None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:16.895262 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854fda8d-b6b9-4d8c-871e-6a9a8894e9ee None admin] PATH_INFO: `/v3/users/nova` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:16.896685 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 314/621] 10.222.0.13 () {56 vars in 1135 bytes} [Mon Jun 27 21:03:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:16.903614 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31a14e5-eb9b-40ba-952e-7a3e4fa25df9 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:16.913954 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854fda8d-b6b9-4d8c-871e-6a9a8894e9ee None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:16.915534 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854fda8d-b6b9-4d8c-871e-6a9a8894e9ee None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:16.941927 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-854fda8d-b6b9-4d8c-871e-6a9a8894e9ee None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 21:03:16.941927 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:16.941927 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:16.941927 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:16.941927 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:16.941927 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:16.941927 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:16.941927 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:16.941927 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:16.941927 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:16.941927 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:16.941927 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:16.941927 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:16.941927 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:16.941927 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:16.941927 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:16.943040 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:16.943040 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:16.943040 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:16.943040 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:16.943040 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:16.943040 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:16.943040 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:16.943040 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:16.943040 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:16.943040 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:16.943040 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:16.943040 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:16.943040 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:16.943040 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:16.943040 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:16.943040 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:16.943040 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:16.943040 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:16.943983 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:16.943983 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:16.943983 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:16.943983 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:16.943983 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:16.943983 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:16.943983 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:16.943983 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:16.943983 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 21:03:16.943983 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:16.953415 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 308/622] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Jun 27 21:03:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:16.968363 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854fda8d-b6b9-4d8c-871e-6a9a8894e9ee None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:17.014371 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31a14e5-eb9b-40ba-952e-7a3e4fa25df9 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:17.090238 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2040b602-f202-4c12-9ff1-93f9d9e2f100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:17.091199 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040b602-f202-4c12-9ff1-93f9d9e2f100 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:17.091544 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040b602-f202-4c12-9ff1-93f9d9e2f100 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:17.091882 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040b602-f202-4c12-9ff1-93f9d9e2f100 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:17.112025 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854fda8d-b6b9-4d8c-871e-6a9a8894e9ee None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:17.125865 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2040b602-f202-4c12-9ff1-93f9d9e2f100 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:17.127460 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2040b602-f202-4c12-9ff1-93f9d9e2f100 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:17.149852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684969b-0ed8-4851-b54e-98fca3904606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:17.150830 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684969b-0ed8-4851-b54e-98fca3904606 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:17.151351 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684969b-0ed8-4851-b54e-98fca3904606 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:17.151802 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684969b-0ed8-4851-b54e-98fca3904606 None admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:17.152852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f684969b-0ed8-4851-b54e-98fca3904606 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:17.153361 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f684969b-0ed8-4851-b54e-98fca3904606 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:17.154947 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f684969b-0ed8-4851-b54e-98fca3904606 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:17.163861 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-2040b602-f202-4c12-9ff1-93f9d9e2f100 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:17.163861 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:17.163861 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:17.163861 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:17.163861 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:17.163861 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:17.163861 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:17.163861 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:17.163861 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:17.163861 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:17.163861 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:17.163861 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:17.163861 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:17.163861 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:17.163861 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:17.164940 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:17.164940 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:17.164940 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:17.164940 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:17.164940 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:17.164940 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:17.164940 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:17.164940 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:17.164940 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:17.164940 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:17.164940 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:17.164940 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:17.164940 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:17.164940 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:17.164940 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:17.164940 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:17.164940 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:17.165851 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:17.165851 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:17.165851 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:17.165851 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:17.170322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 315/623] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:17.174041 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2040b602-f202-4c12-9ff1-93f9d9e2f100 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:17.178434 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 309/624] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Jun 27 21:03:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:17.189020 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684969b-0ed8-4851-b54e-98fca3904606 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:17.238496 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:17.269257 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:17.335682 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684969b-0ed8-4851-b54e-98fca3904606 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:17.375173 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60534e9-8b95-4939-b9a4-f9e0e08e4c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:17.375843 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60534e9-8b95-4939-b9a4-f9e0e08e4c55 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:17.376122 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60534e9-8b95-4939-b9a4-f9e0e08e4c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:17.376395 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60534e9-8b95-4939-b9a4-f9e0e08e4c55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:17.387118 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2040b602-f202-4c12-9ff1-93f9d9e2f100 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:17.434557 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7dff89-bcf2-4e75-a7e8-1bf2a80d3a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:17.435230 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7dff89-bcf2-4e75-a7e8-1bf2a80d3a1b None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:17.435513 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7dff89-bcf2-4e75-a7e8-1bf2a80d3a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:17.436136 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7dff89-bcf2-4e75-a7e8-1bf2a80d3a1b None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:17.436619 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7dff89-bcf2-4e75-a7e8-1bf2a80d3a1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:17.436956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7dff89-bcf2-4e75-a7e8-1bf2a80d3a1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:17.438531 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7dff89-bcf2-4e75-a7e8-1bf2a80d3a1b None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:17.452409 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 316/625] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:17.458176 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad11ed-aaad-4924-8f3b-3e0cf66ca1b1 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:17.458467 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad11ed-aaad-4924-8f3b-3e0cf66ca1b1 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:17.458745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad11ed-aaad-4924-8f3b-3e0cf66ca1b1 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:17.464285 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 317/626] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:17.464285 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ad11ed-aaad-4924-8f3b-3e0cf66ca1b1 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:17.475460 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60534e9-8b95-4939-b9a4-f9e0e08e4c55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:17.478421 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60534e9-8b95-4939-b9a4-f9e0e08e4c55 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:17.533621 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:17.597098 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:17.608445 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ad11ed-aaad-4924-8f3b-3e0cf66ca1b1 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:17.619023 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 310/627] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 21:03:17.621012 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41ea6d-7880-4328-a584-30446d035003 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:17.621499 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41ea6d-7880-4328-a584-30446d035003 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:17.621963 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41ea6d-7880-4328-a584-30446d035003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:17.639004 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-7a41ea6d-7880-4328-a584-30446d035003 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:17.640673 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-7a41ea6d-7880-4328-a584-30446d035003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:17.651399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a5c6a3-ae59-40ff-b96b-d78bd9c614f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:17.652643 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5c6a3-ae59-40ff-b96b-d78bd9c614f7 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:17.652946 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5c6a3-ae59-40ff-b96b-d78bd9c614f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:17.653374 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a5c6a3-ae59-40ff-b96b-d78bd9c614f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:17.660784 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a5c6a3-ae59-40ff-b96b-d78bd9c614f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:17.662251 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a5c6a3-ae59-40ff-b96b-d78bd9c614f7 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:17.676419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-59a5c6a3-ae59-40ff-b96b-d78bd9c614f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:17.676419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:17.676419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:17.676419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:17.676419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:17.676419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:17.676419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:17.676419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:17.676419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:17.676419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:17.676419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:17.676419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:17.676419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:17.676419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:17.676419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:17.677378 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:17.677378 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:17.677378 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:17.677378 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:17.677378 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:17.677378 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:17.677378 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:17.677378 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:17.677378 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:17.677378 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:17.677378 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:17.677378 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:17.677378 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:17.677378 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:17.677378 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:17.677378 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:17.677378 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:17.678346 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:17.678346 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:17.678346 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:17.678346 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:17.678346 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:17.678346 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:17.678346 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:17.678346 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:17.678346 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 318/628] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:17.679258 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e9647-55c8-4073-971d-edd4f62e5f83 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:17.679431 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e9647-55c8-4073-971d-edd4f62e5f83 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:17.679701 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e9647-55c8-4073-971d-edd4f62e5f83 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:17.681092 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 319/629] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:17.685595 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e9647-55c8-4073-971d-edd4f62e5f83 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:17.714537 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 311/630] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:17.715487 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a41ea6d-7880-4328-a584-30446d035003 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:17.792837 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:17.816499 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e9647-55c8-4073-971d-edd4f62e5f83 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:17.824309 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a41ea6d-7880-4328-a584-30446d035003 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:17.857055 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae0006-36f8-4490-a3f2-bb9d8c1fafd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:17.857861 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae0006-36f8-4490-a3f2-bb9d8c1fafd1 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:17.858147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae0006-36f8-4490-a3f2-bb9d8c1fafd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:17.858423 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae0006-36f8-4490-a3f2-bb9d8c1fafd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:17.865032 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8316f894-1c77-4cba-bf46-a03f9b98d836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:17.865706 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316f894-1c77-4cba-bf46-a03f9b98d836 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:17.865986 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316f894-1c77-4cba-bf46-a03f9b98d836 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:17.866259 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316f894-1c77-4cba-bf46-a03f9b98d836 None admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:17.867052 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8316f894-1c77-4cba-bf46-a03f9b98d836 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:17.867353 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8316f894-1c77-4cba-bf46-a03f9b98d836 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:17.868786 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8316f894-1c77-4cba-bf46-a03f9b98d836 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:17.871035 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ae0006-36f8-4490-a3f2-bb9d8c1fafd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:17.872753 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ae0006-36f8-4490-a3f2-bb9d8c1fafd1 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:17.885884 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 312/631] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:17.887641 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff660c5-4e81-448f-813b-896081ba16eb None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:17.887925 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff660c5-4e81-448f-813b-896081ba16eb None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:17.888204 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff660c5-4e81-448f-813b-896081ba16eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:17.910481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-c6ae0006-36f8-4490-a3f2-bb9d8c1fafd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:17.910481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:17.910481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:17.910481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:17.910481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:17.910481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:17.910481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:17.910481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:17.910481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:17.910481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:17.910481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:17.910481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:17.910481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:17.910481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:17.910481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:17.911473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:17.911473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:17.911473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:17.911473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:17.911473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:17.911473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:17.911473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:17.911473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:17.911473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:17.911473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:17.911473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:17.911473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:17.911473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:17.911473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:17.911473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:17.911473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:17.911473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:17.912352 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:17.912352 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:17.912352 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:17.912352 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:17.912583 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-eff660c5-4e81-448f-813b-896081ba16eb None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:17.912583 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-eff660c5-4e81-448f-813b-896081ba16eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:17.916288 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 320/632] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:17.917207 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae0006-36f8-4490-a3f2-bb9d8c1fafd1 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:17.929422 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:17.997147 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 313/633] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:17.997147 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff660c5-4e81-448f-813b-896081ba16eb None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:18.046500 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae0006-36f8-4490-a3f2-bb9d8c1fafd1 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:18.074318 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:18.082132 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c37b3-b01c-4c32-a9ca-9539f8a39ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:18.083082 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c37b3-b01c-4c32-a9ca-9539f8a39ce6 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:18.083366 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c37b3-b01c-4c32-a9ca-9539f8a39ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:18.083643 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c37b3-b01c-4c32-a9ca-9539f8a39ce6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:18.095205 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393c37b3-b01c-4c32-a9ca-9539f8a39ce6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:18.103367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-393c37b3-b01c-4c32-a9ca-9539f8a39ce6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:18.103367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:18.103367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:18.103367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:18.103367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:18.103367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:18.103367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:18.103367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:18.103367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:18.103367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:18.103367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:18.103367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:18.103367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:18.103367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:18.103367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:18.103367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:18.109681 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:18.109681 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:18.109681 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:18.109681 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:18.109681 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:18.109681 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:18.109681 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:18.109681 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:18.109681 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:18.109681 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:18.109681 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:18.109681 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:18.109681 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:18.109681 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:18.109681 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:18.109681 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:18.110613 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:18.110613 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:18.110613 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:18.110613 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:18.110613 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:18.110613 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:18.110613 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:18.110613 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:18.110613 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:18.110613 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:18.110613 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:18.110613 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:18.110613 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 321/634] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 27 21:03:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:18.110613 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c37b3-b01c-4c32-a9ca-9539f8a39ce6 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:18.124856 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff660c5-4e81-448f-813b-896081ba16eb None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:18.164809 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15df5d43-2b69-44a3-a565-b667ecd0d5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:18.165463 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df5d43-2b69-44a3-a565-b667ecd0d5aa None admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:18.165805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df5d43-2b69-44a3-a565-b667ecd0d5aa None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:18.166136 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df5d43-2b69-44a3-a565-b667ecd0d5aa None admin] PATH_INFO: `/v3/projects/334438c929c44e489e9a3b93e6d37231/users/8bed6b231ee647d8a350da055d8e59d7/roles/709aef6b85524eb2a8e9986a2bd9525b` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:18.183170 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15df5d43-2b69-44a3-a565-b667ecd0d5aa None admin] RBAC: Authorizing `identity:create_grant(project_id=334438c929c44e489e9a3b93e6d37231, user_id=8bed6b231ee647d8a350da055d8e59d7, role_id=709aef6b85524eb2a8e9986a2bd9525b)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:18.184647 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15df5d43-2b69-44a3-a565-b667ecd0d5aa None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:18.199433 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:18.203691 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 314/635] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 27 21:03:17 2022] PUT /identity/v3/projects/334438c929c44e489e9a3b93e6d37231/users/8bed6b231ee647d8a350da055d8e59d7/roles/709aef6b85524eb2a8e9986a2bd9525b => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:03:18.205043 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15df5d43-2b69-44a3-a565-b667ecd0d5aa None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:18.266172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c37b3-b01c-4c32-a9ca-9539f8a39ce6 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:18.315644 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2da76a3-cd8b-4c0f-a911-24f4ec26037c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:18.316236 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da76a3-cd8b-4c0f-a911-24f4ec26037c None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:18.316561 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da76a3-cd8b-4c0f-a911-24f4ec26037c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:18.316836 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da76a3-cd8b-4c0f-a911-24f4ec26037c None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:18.317609 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2da76a3-cd8b-4c0f-a911-24f4ec26037c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:18.317913 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2da76a3-cd8b-4c0f-a911-24f4ec26037c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:18.319735 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2da76a3-cd8b-4c0f-a911-24f4ec26037c None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:18.338088 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 322/636] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:18.339876 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2da76a3-cd8b-4c0f-a911-24f4ec26037c None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:18.359237 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15df5d43-2b69-44a3-a565-b667ecd0d5aa None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:18.377773 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:18.412462 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc383e8-965b-4acf-a46f-082c524c3ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:18.413420 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc383e8-965b-4acf-a46f-082c524c3ecb None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:18.413705 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc383e8-965b-4acf-a46f-082c524c3ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:18.414096 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc383e8-965b-4acf-a46f-082c524c3ecb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:18.427582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc383e8-965b-4acf-a46f-082c524c3ecb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:18.437528 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-dbc383e8-965b-4acf-a46f-082c524c3ecb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:18.437528 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:18.437528 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:18.437528 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:18.437528 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:18.437528 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:18.437528 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:18.437528 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:18.437528 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:18.437528 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:18.437528 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:18.437528 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:18.437528 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:18.437528 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:18.437528 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:18.437528 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:18.438953 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:18.438953 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:18.438953 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:18.438953 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:18.438953 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:18.438953 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:18.438953 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:18.438953 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:18.438953 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:18.438953 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:18.438953 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:18.438953 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:18.438953 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:18.438953 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:18.438953 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:18.438953 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:18.440625 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:18.440625 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:18.440625 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:18.440625 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:18.440625 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:18.440625 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:18.440625 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:18.440625 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:18.440625 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:18.440625 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:18.440625 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:18.440625 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:18.440625 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 315/637] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 27 21:03:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:18.440625 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc383e8-965b-4acf-a46f-082c524c3ecb None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:18.466607 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2da76a3-cd8b-4c0f-a911-24f4ec26037c None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:18.498899 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:18.505454 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5312c66c-ab77-4153-8cdd-a5aee6e13e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:18.506128 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312c66c-ab77-4153-8cdd-a5aee6e13e63 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:18.506414 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312c66c-ab77-4153-8cdd-a5aee6e13e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:18.506685 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312c66c-ab77-4153-8cdd-a5aee6e13e63 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:18.507447 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5312c66c-ab77-4153-8cdd-a5aee6e13e63 None admin] RBAC: Adding query filter params (name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:18.507757 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5312c66c-ab77-4153-8cdd-a5aee6e13e63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:18.509174 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5312c66c-ab77-4153-8cdd-a5aee6e13e63 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:18.522597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 323/638] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 27 21:03:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:18.522597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62131552-5374-498d-8289-50a4d9069f0c None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:18.522597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62131552-5374-498d-8289-50a4d9069f0c None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:18.522597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62131552-5374-498d-8289-50a4d9069f0c None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:18.522597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 324/639] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:18.522597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62131552-5374-498d-8289-50a4d9069f0c None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:18.561788 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc383e8-965b-4acf-a46f-082c524c3ecb None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:18.596247 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2625b9c6-fa67-4c05-99e2-bf804c727d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:18.596997 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625b9c6-fa67-4c05-99e2-bf804c727d15 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:18.597282 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625b9c6-fa67-4c05-99e2-bf804c727d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:18.597558 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2625b9c6-fa67-4c05-99e2-bf804c727d15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:18.605286 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2625b9c6-fa67-4c05-99e2-bf804c727d15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:18.606802 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2625b9c6-fa67-4c05-99e2-bf804c727d15 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:18.621466 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-2625b9c6-fa67-4c05-99e2-bf804c727d15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:18.621466 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:18.621466 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:18.621466 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:18.621466 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:18.621466 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:18.621466 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:18.621466 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:18.621466 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:18.621466 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:18.621466 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:18.621466 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:18.621466 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:18.621466 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:18.621466 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:18.622417 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:18.622417 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:18.622417 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:18.622417 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:18.622417 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:18.622417 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:18.622417 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:18.622417 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:18.622417 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:18.622417 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:18.622417 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:18.622417 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:18.622417 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:18.622417 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:18.622417 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:18.622417 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:18.622417 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:18.623300 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:18.623300 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:18.623300 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:18.623300 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:18.623300 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:18.623300 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:18.623300 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:18.623300 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:18.627489 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 316/640] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:18.628116 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2625b9c6-fa67-4c05-99e2-bf804c727d15 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:18.635120 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62131552-5374-498d-8289-50a4d9069f0c None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:18.665063 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:18.676392 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f6a211-1edd-4ab2-91ef-09cc29639e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:18.677325 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6a211-1edd-4ab2-91ef-09cc29639e66 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:18.677545 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6a211-1edd-4ab2-91ef-09cc29639e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:18.677767 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6a211-1edd-4ab2-91ef-09cc29639e66 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:18.678646 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f6a211-1edd-4ab2-91ef-09cc29639e66 None admin] RBAC: Adding query filter params (name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:18.678950 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f6a211-1edd-4ab2-91ef-09cc29639e66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:18.680957 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f6a211-1edd-4ab2-91ef-09cc29639e66 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:18.693278 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 325/641] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 27 21:03:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:18.695912 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69c4bb-2d90-49f6-abd1-37a7d669fe5b None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:18.696206 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69c4bb-2d90-49f6-abd1-37a7d669fe5b None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:18.696482 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69c4bb-2d90-49f6-abd1-37a7d669fe5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:18.714494 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-be69c4bb-2d90-49f6-abd1-37a7d669fe5b None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:18.715707 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-be69c4bb-2d90-49f6-abd1-37a7d669fe5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:18.748654 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2625b9c6-fa67-4c05-99e2-bf804c727d15 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:18.770641 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:18.787919 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa9aa9-e35a-4cab-b012-d994137746ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:18.788806 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa9aa9-e35a-4cab-b012-d994137746ba None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:18.789120 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa9aa9-e35a-4cab-b012-d994137746ba None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:18.789397 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa9aa9-e35a-4cab-b012-d994137746ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:18.804968 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 326/642] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:18.804968 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69c4bb-2d90-49f6-abd1-37a7d669fe5b None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:18.815014 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affa9aa9-e35a-4cab-b012-d994137746ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:18.816703 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affa9aa9-e35a-4cab-b012-d994137746ba None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:18.836423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-affa9aa9-e35a-4cab-b012-d994137746ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:18.836423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:18.836423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:18.836423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:18.836423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:18.836423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:18.836423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:18.836423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:18.836423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:18.836423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:18.836423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:18.836423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:18.836423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:18.836423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:18.836423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:18.837782 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:18.837782 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:18.837782 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:18.837782 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:18.837782 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:18.837782 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:18.837782 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:18.837782 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:18.837782 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:18.837782 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:18.837782 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:18.837782 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:18.837782 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:18.837782 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:18.837782 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:18.837782 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:18.837782 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:18.839327 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:18.839327 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:18.839327 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:18.839327 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:18.842716 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 317/643] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:18.843613 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa9aa9-e35a-4cab-b012-d994137746ba None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:18.919893 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69c4bb-2d90-49f6-abd1-37a7d669fe5b None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:18.930074 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:18.951615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa9aa9-e35a-4cab-b012-d994137746ba None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:18.962851 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102a0f7-c7ed-4368-92c3-776ff3c19e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:18.963525 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102a0f7-c7ed-4368-92c3-776ff3c19e76 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:18.964019 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102a0f7-c7ed-4368-92c3-776ff3c19e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:18.964796 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a102a0f7-c7ed-4368-92c3-776ff3c19e76 None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:18.965637 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102a0f7-c7ed-4368-92c3-776ff3c19e76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:18.965946 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102a0f7-c7ed-4368-92c3-776ff3c19e76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:18.967393 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a102a0f7-c7ed-4368-92c3-776ff3c19e76 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:18.980663 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 327/644] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:18.981468 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102a0f7-c7ed-4368-92c3-776ff3c19e76 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:18.994660 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a9b370-6107-4149-b9ae-dbe1ed0d2023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:18.996152 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a9b370-6107-4149-b9ae-dbe1ed0d2023 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:18.996435 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a9b370-6107-4149-b9ae-dbe1ed0d2023 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:18.996707 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a9b370-6107-4149-b9ae-dbe1ed0d2023 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:19.007131 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a9b370-6107-4149-b9ae-dbe1ed0d2023 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:19.008575 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a9b370-6107-4149-b9ae-dbe1ed0d2023 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:19.030341 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-57a9b370-6107-4149-b9ae-dbe1ed0d2023 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:19.030341 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:19.030341 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:19.030341 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:19.030341 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:19.030341 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:19.030341 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:19.030341 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:19.030341 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:19.030341 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:19.030341 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:19.030341 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:19.030341 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:19.030341 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:19.030341 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:19.031292 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:19.031292 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:19.031292 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:19.031292 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:19.031292 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:19.031292 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:19.031292 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:19.031292 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:19.031292 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:19.031292 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:19.031292 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:19.031292 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:19.031292 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:19.031292 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:19.031292 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:19.031292 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:19.031292 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:19.032328 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:19.032328 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:19.032328 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:19.032328 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:19.033068 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 318/645] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:19.033883 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a9b370-6107-4149-b9ae-dbe1ed0d2023 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:19.045184 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:19.106600 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a102a0f7-c7ed-4368-92c3-776ff3c19e76 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:19.145627 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a9b370-6107-4149-b9ae-dbe1ed0d2023 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:19.146363 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13222cfd-3b8b-4dc1-b99e-bea77b6a100a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:19.147194 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13222cfd-3b8b-4dc1-b99e-bea77b6a100a None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:19.147470 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13222cfd-3b8b-4dc1-b99e-bea77b6a100a None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:19.147741 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13222cfd-3b8b-4dc1-b99e-bea77b6a100a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:19.160272 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13222cfd-3b8b-4dc1-b99e-bea77b6a100a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:19.169628 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-13222cfd-3b8b-4dc1-b99e-bea77b6a100a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:19.169628 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:19.169628 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:19.169628 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:19.169628 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:19.169628 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:19.169628 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:19.169628 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:19.169628 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:19.169628 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:19.169628 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:19.169628 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:19.169628 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:19.169628 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:19.169628 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:19.169628 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:19.179368 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:19.179368 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:19.179368 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:19.179368 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:19.179368 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:19.179368 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:19.179368 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:19.179368 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:19.179368 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:19.179368 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:19.179368 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:19.179368 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:19.179368 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:19.179368 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:19.179368 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:19.179368 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:19.180225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:19.180225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:19.180225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:19.180225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:19.180225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:19.180225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:19.180225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:19.180225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:19.180225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:19.180225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:19.180225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:19.180225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:19.180225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 328/646] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 27 21:03:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:19.180225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13222cfd-3b8b-4dc1-b99e-bea77b6a100a None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:19.199052 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47496a71-7459-4886-b638-53bcaee2327d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:19.199802 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47496a71-7459-4886-b638-53bcaee2327d None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:19.200083 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47496a71-7459-4886-b638-53bcaee2327d None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:19.200354 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47496a71-7459-4886-b638-53bcaee2327d None admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:19.201122 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47496a71-7459-4886-b638-53bcaee2327d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:19.201425 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47496a71-7459-4886-b638-53bcaee2327d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:19.202869 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47496a71-7459-4886-b638-53bcaee2327d None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:19.204511 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:19.216664 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 319/647] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:19.217410 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47496a71-7459-4886-b638-53bcaee2327d None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:19.307218 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13222cfd-3b8b-4dc1-b99e-bea77b6a100a None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:19.310544 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:19.326145 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47496a71-7459-4886-b638-53bcaee2327d None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:19.344532 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19446547-f884-45d7-9e4c-7c9ffc63a582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:19.345225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19446547-f884-45d7-9e4c-7c9ffc63a582 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:19.345507 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19446547-f884-45d7-9e4c-7c9ffc63a582 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:19.345779 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19446547-f884-45d7-9e4c-7c9ffc63a582 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:19.346619 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19446547-f884-45d7-9e4c-7c9ffc63a582 None admin] RBAC: Adding query filter params (role.id=709aef6b85524eb2a8e9986a2bd9525b, scope.project.id=334438c929c44e489e9a3b93e6d37231, user.id=1366958034d540b4ba2feb421a6afabc) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:19.346929 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19446547-f884-45d7-9e4c-7c9ffc63a582 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:19.348365 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19446547-f884-45d7-9e4c-7c9ffc63a582 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:19.360989 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 329/648] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:03:19 2022] GET /identity/v3/role_assignments?user.id=1366958034d540b4ba2feb421a6afabc&scope.project.id=334438c929c44e489e9a3b93e6d37231&role.id=709aef6b85524eb2a8e9986a2bd9525b => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:19.360989 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19446547-f884-45d7-9e4c-7c9ffc63a582 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:19.367599 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005e785-c05e-4024-b1e2-1b87332d51e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:19.368420 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005e785-c05e-4024-b1e2-1b87332d51e7 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:19.368777 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005e785-c05e-4024-b1e2-1b87332d51e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:19.369156 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005e785-c05e-4024-b1e2-1b87332d51e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:19.370117 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1005e785-c05e-4024-b1e2-1b87332d51e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:19.370698 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1005e785-c05e-4024-b1e2-1b87332d51e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:19.372159 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1005e785-c05e-4024-b1e2-1b87332d51e7 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:19.391548 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 320/649] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:19.392647 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005e785-c05e-4024-b1e2-1b87332d51e7 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:19.466963 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:19.502835 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19446547-f884-45d7-9e4c-7c9ffc63a582 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:19.516612 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005e785-c05e-4024-b1e2-1b87332d51e7 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:19.548242 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ba654-3065-4d8d-9ff9-180a4205d813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:19.549055 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ba654-3065-4d8d-9ff9-180a4205d813 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:19.549469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ba654-3065-4d8d-9ff9-180a4205d813 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:19.549815 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ba654-3065-4d8d-9ff9-180a4205d813 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:19.550653 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ba654-3065-4d8d-9ff9-180a4205d813 None admin] RBAC: Adding query filter params (name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:19.551705 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ba654-3065-4d8d-9ff9-180a4205d813 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:19.552506 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ba654-3065-4d8d-9ff9-180a4205d813 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:19.559839 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2572d287-c04a-40d1-8d89-17375341b6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:19.561009 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572d287-c04a-40d1-8d89-17375341b6b1 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:19.561511 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572d287-c04a-40d1-8d89-17375341b6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:19.561982 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572d287-c04a-40d1-8d89-17375341b6b1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:19.562701 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 330/650] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 27 21:03:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:19.563545 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ba654-3065-4d8d-9ff9-180a4205d813 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:19.570889 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2572d287-c04a-40d1-8d89-17375341b6b1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:19.572771 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2572d287-c04a-40d1-8d89-17375341b6b1 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:19.590580 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-2572d287-c04a-40d1-8d89-17375341b6b1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 21:03:19.590580 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:19.590580 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:19.590580 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:19.590580 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:19.590580 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:19.590580 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:19.590580 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:19.590580 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:19.590580 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:19.590580 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:19.590580 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:19.590580 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:19.590580 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:19.590580 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:19.590580 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:19.591578 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:19.591578 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:19.591578 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:19.591578 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:19.591578 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:19.591578 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:19.591578 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:19.591578 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:19.591578 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:19.591578 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:19.591578 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:19.591578 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:19.591578 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:19.591578 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:19.591578 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:19.591578 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:19.591578 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:19.591578 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:19.592544 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:19.592544 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:19.592544 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:19.592544 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:19.592544 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:19.592544 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:19.592544 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:19.592544 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:19.592544 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 21:03:19.592544 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:19.596053 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 321/651] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:03:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:19.596955 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2572d287-c04a-40d1-8d89-17375341b6b1 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:19.599297 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:19.684584 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ba654-3065-4d8d-9ff9-180a4205d813 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:19.724185 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2572d287-c04a-40d1-8d89-17375341b6b1 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:19.735246 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a97bb7-2cb1-4865-9f38-ddd68bb310fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:19.737539 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a97bb7-2cb1-4865-9f38-ddd68bb310fa None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:19.738681 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a97bb7-2cb1-4865-9f38-ddd68bb310fa None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:19.739780 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a97bb7-2cb1-4865-9f38-ddd68bb310fa None admin] PATH_INFO: `/v3/users/glance` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:19.742265 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:19.762599 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a97bb7-2cb1-4865-9f38-ddd68bb310fa None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:19.766832 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a97bb7-2cb1-4865-9f38-ddd68bb310fa None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:19.769261 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45229141-0042-4888-acfd-27705ff58772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:19.770149 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45229141-0042-4888-acfd-27705ff58772 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:19.770438 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45229141-0042-4888-acfd-27705ff58772 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:19.770711 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45229141-0042-4888-acfd-27705ff58772 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:19.784692 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45229141-0042-4888-acfd-27705ff58772 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:19.786192 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45229141-0042-4888-acfd-27705ff58772 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:19.799810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-53a97bb7-2cb1-4865-9f38-ddd68bb310fa None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 27 21:03:19.799810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:19.799810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:19.799810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:19.799810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:19.799810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:19.799810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:19.799810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:19.799810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:19.799810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:19.799810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:19.799810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:19.799810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:19.799810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:19.799810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:19.799810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:19.800831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:19.800831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:19.800831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:19.800831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:19.800831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:19.800831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:19.800831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:19.800831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:19.800831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:19.800831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:19.800831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:19.800831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:19.800831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:19.800831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:19.800831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:19.800831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:19.800831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:19.800831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:19.801816 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:19.801816 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:19.801816 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:19.801816 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:19.801816 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:19.801816 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:19.801816 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:19.801816 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:19.801816 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 27 21:03:19.801816 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:19.804509 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 331/652] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:03:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:19.805569 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-45229141-0042-4888-acfd-27705ff58772 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:19.805569 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:19.805569 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:19.805569 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:19.805569 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:19.805569 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:19.805569 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:19.805569 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:19.805569 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:19.805569 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:19.805569 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:19.805569 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:19.805569 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:19.805569 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:19.805569 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:19.806534 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:19.806534 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:19.806534 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:19.806534 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:19.806534 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:19.806534 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:19.806534 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:19.806534 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:19.806534 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:19.806534 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:19.806534 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:19.806534 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:19.806534 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:19.806534 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:19.806534 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:19.806534 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:19.806534 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:19.807416 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:19.807416 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:19.807416 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:19.807416 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:19.807416 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 322/653] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:19.808310 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a97bb7-2cb1-4865-9f38-ddd68bb310fa None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:19.812654 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45229141-0042-4888-acfd-27705ff58772 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:19.927131 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:19.941007 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a97bb7-2cb1-4865-9f38-ddd68bb310fa None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:19.949808 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45229141-0042-4888-acfd-27705ff58772 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:19.977504 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37f2b37-b95a-472f-844e-cecbf75ca7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:19.978086 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f2b37-b95a-472f-844e-cecbf75ca7e8 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:19.978366 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f2b37-b95a-472f-844e-cecbf75ca7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:19.978638 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f2b37-b95a-472f-844e-cecbf75ca7e8 None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:19.979767 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37f2b37-b95a-472f-844e-cecbf75ca7e8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:19.979899 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37f2b37-b95a-472f-844e-cecbf75ca7e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:19.981543 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37f2b37-b95a-472f-844e-cecbf75ca7e8 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:19.994404 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d618f3-61a0-4d79-bb87-acf3d8fc32fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:19.995672 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d618f3-61a0-4d79-bb87-acf3d8fc32fa None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:19.996364 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d618f3-61a0-4d79-bb87-acf3d8fc32fa None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:19.996864 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d618f3-61a0-4d79-bb87-acf3d8fc32fa None admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:19.998028 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d618f3-61a0-4d79-bb87-acf3d8fc32fa None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:19.998557 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d618f3-61a0-4d79-bb87-acf3d8fc32fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:20.000253 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d618f3-61a0-4d79-bb87-acf3d8fc32fa None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:20.005028 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 332/654] 10.222.0.13 () {56 vars in 1137 bytes} [Mon Jun 27 21:03:19 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:20.005028 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37f2b37-b95a-472f-844e-cecbf75ca7e8 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:20.031695 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 323/655] 10.222.0.13 () {56 vars in 1137 bytes} [Mon Jun 27 21:03:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:20.032595 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d618f3-61a0-4d79-bb87-acf3d8fc32fa None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:20.067865 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:20.146788 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37f2b37-b95a-472f-844e-cecbf75ca7e8 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:20.182471 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15425fa-4669-42a4-80da-78c9f75d990e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:20.183165 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15425fa-4669-42a4-80da-78c9f75d990e None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:20.183437 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15425fa-4669-42a4-80da-78c9f75d990e None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:20.183804 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15425fa-4669-42a4-80da-78c9f75d990e None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:20.184516 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15425fa-4669-42a4-80da-78c9f75d990e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:20.184755 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15425fa-4669-42a4-80da-78c9f75d990e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:20.186134 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15425fa-4669-42a4-80da-78c9f75d990e None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:20.202541 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 333/656] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:20.203456 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15425fa-4669-42a4-80da-78c9f75d990e None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:20.215194 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:20.220022 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d618f3-61a0-4d79-bb87-acf3d8fc32fa None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:20.283935 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb75f612-d155-4e37-8286-c361636e88c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:20.285559 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75f612-d155-4e37-8286-c361636e88c9 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:20.286375 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75f612-d155-4e37-8286-c361636e88c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:20.287073 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75f612-d155-4e37-8286-c361636e88c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:20.308318 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15425fa-4669-42a4-80da-78c9f75d990e None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:20.346095 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6013b-1784-4f16-8bf7-8799a88eb3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:20.347456 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6013b-1784-4f16-8bf7-8799a88eb3d6 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:20.348239 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6013b-1784-4f16-8bf7-8799a88eb3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:20.349018 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6013b-1784-4f16-8bf7-8799a88eb3d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:20.376380 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb75f612-d155-4e37-8286-c361636e88c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:20.377874 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb75f612-d155-4e37-8286-c361636e88c9 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:20.403645 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:20.439311 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6013b-1784-4f16-8bf7-8799a88eb3d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:20.441922 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6013b-1784-4f16-8bf7-8799a88eb3d6 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:20.455463 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:20.488602 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 324/657] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:03:20.489568 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb75f612-d155-4e37-8286-c361636e88c9 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:20.569291 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 334/658] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:03:20.569291 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bfbc1-4206-4481-8c95-8925b42532d6 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:20.569291 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bfbc1-4206-4481-8c95-8925b42532d6 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:20.569291 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bfbc1-4206-4481-8c95-8925b42532d6 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:20.569291 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 335/659] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:20.569291 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9bfbc1-4206-4481-8c95-8925b42532d6 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:20.625553 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb75f612-d155-4e37-8286-c361636e88c9 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:20.654547 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:20.668642 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659d145-aacd-4716-ac37-b54ab2b13247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:20.669502 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659d145-aacd-4716-ac37-b54ab2b13247 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:20.669781 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659d145-aacd-4716-ac37-b54ab2b13247 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:20.670055 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659d145-aacd-4716-ac37-b54ab2b13247 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:20.679123 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3659d145-aacd-4716-ac37-b54ab2b13247 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:20.680853 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3659d145-aacd-4716-ac37-b54ab2b13247 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:20.685962 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9bfbc1-4206-4481-8c95-8925b42532d6 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:20.696591 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:20.696769 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-3659d145-aacd-4716-ac37-b54ab2b13247 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 27 21:03:20.696769 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:20.696769 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:20.696769 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:20.696769 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:20.696769 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:20.696769 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:20.696769 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:20.696769 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:20.696769 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:20.696769 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:20.696769 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:20.696769 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:20.696769 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:20.696769 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:20.696769 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:20.697788 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:20.697788 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:20.697788 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:20.697788 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:20.697788 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:20.697788 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:20.697788 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:20.697788 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:20.697788 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:20.697788 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:20.697788 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:20.697788 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:20.697788 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:20.697788 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:20.697788 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:20.697788 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:20.697788 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:20.697788 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:20.699243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:20.699243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:20.699243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:20.699243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:20.699243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:20.699243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:20.699243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:20.699243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:20.699243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 27 21:03:20.699243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:20.699243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 325/660] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 27 21:03:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:20.700878 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659d145-aacd-4716-ac37-b54ab2b13247 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:20.731406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c32f9-dbe9-4559-9e4c-da4c537bfcc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:20.732261 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c32f9-dbe9-4559-9e4c-da4c537bfcc2 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:20.732541 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c32f9-dbe9-4559-9e4c-da4c537bfcc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:20.732818 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c32f9-dbe9-4559-9e4c-da4c537bfcc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:20.743182 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c32f9-dbe9-4559-9e4c-da4c537bfcc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:20.744646 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c32f9-dbe9-4559-9e4c-da4c537bfcc2 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:20.763011 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-a62c32f9-dbe9-4559-9e4c-da4c537bfcc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:20.763011 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:20.763011 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:20.763011 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:20.763011 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:20.763011 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:20.763011 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:20.763011 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:20.763011 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:20.763011 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:20.763011 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:20.763011 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:20.763011 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:20.763011 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:20.763011 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:20.764172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:20.764172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:20.764172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:20.764172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:20.764172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:20.764172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:20.764172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:20.764172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:20.764172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:20.764172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:20.764172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:20.764172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:20.764172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:20.764172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:20.764172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:20.764172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:20.764172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:20.765679 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:20.765679 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:20.765679 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:20.765679 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:20.768833 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 336/661] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:20.770045 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaccc88-0c6b-404f-ae93-ec5dbfb504fa None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:20.770327 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaccc88-0c6b-404f-ae93-ec5dbfb504fa None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:20.770600 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaccc88-0c6b-404f-ae93-ec5dbfb504fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:20.789659 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-8eaccc88-0c6b-404f-ae93-ec5dbfb504fa None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:20.790890 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-8eaccc88-0c6b-404f-ae93-ec5dbfb504fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:20.819944 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659d145-aacd-4716-ac37-b54ab2b13247 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:20.865107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98764f0c-4210-4b22-8190-867b77dde6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:20.866762 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98764f0c-4210-4b22-8190-867b77dde6bc None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:20.867057 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98764f0c-4210-4b22-8190-867b77dde6bc None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:20.868009 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98764f0c-4210-4b22-8190-867b77dde6bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:20.881385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 337/662] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:20.881385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaccc88-0c6b-404f-ae93-ec5dbfb504fa None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:20.888239 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98764f0c-4210-4b22-8190-867b77dde6bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:20.891303 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98764f0c-4210-4b22-8190-867b77dde6bc None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:20.924436 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-98764f0c-4210-4b22-8190-867b77dde6bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:20.924436 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:20.924436 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:20.924436 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:20.924436 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:20.924436 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:20.924436 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:20.924436 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:20.924436 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:20.924436 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:20.924436 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:20.924436 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:20.924436 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:20.924436 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:20.924436 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:20.925709 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:20.925709 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:20.925709 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:20.925709 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:20.925709 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:20.925709 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:20.925709 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:20.925709 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:20.925709 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:20.925709 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:20.925709 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:20.925709 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:20.925709 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:20.925709 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:20.925709 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:20.925709 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:20.925709 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:20.926725 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:20.926725 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:20.926725 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:20.926725 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:20.926725 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 326/663] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:20.927462 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98764f0c-4210-4b22-8190-867b77dde6bc None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:20.969390 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:20.990950 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:21.001809 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaccc88-0c6b-404f-ae93-ec5dbfb504fa None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:21.038517 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9caa69-a5e4-4317-8cab-489abb69dd82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:21.039326 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9caa69-a5e4-4317-8cab-489abb69dd82 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:21.039660 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9caa69-a5e4-4317-8cab-489abb69dd82 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:21.039997 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9caa69-a5e4-4317-8cab-489abb69dd82 None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:21.040906 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9caa69-a5e4-4317-8cab-489abb69dd82 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:21.041304 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9caa69-a5e4-4317-8cab-489abb69dd82 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:21.042824 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9caa69-a5e4-4317-8cab-489abb69dd82 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:21.051142 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98764f0c-4210-4b22-8190-867b77dde6bc None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:21.077481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 338/664] 10.222.0.13 () {56 vars in 1139 bytes} [Mon Jun 27 21:03:20 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:21.077481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9caa69-a5e4-4317-8cab-489abb69dd82 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:21.092954 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21e9fbc-cb9a-4a5f-95e3-8ac22389dc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:21.093744 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e9fbc-cb9a-4a5f-95e3-8ac22389dc21 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:21.094094 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e9fbc-cb9a-4a5f-95e3-8ac22389dc21 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:21.094718 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e9fbc-cb9a-4a5f-95e3-8ac22389dc21 None admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:21.095166 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21e9fbc-cb9a-4a5f-95e3-8ac22389dc21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:21.095475 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21e9fbc-cb9a-4a5f-95e3-8ac22389dc21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:21.096868 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21e9fbc-cb9a-4a5f-95e3-8ac22389dc21 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:21.113569 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 327/665] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:21.113569 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21e9fbc-cb9a-4a5f-95e3-8ac22389dc21 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:21.216163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:21.223092 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9caa69-a5e4-4317-8cab-489abb69dd82 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:21.223315 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21e9fbc-cb9a-4a5f-95e3-8ac22389dc21 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:21.262090 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68166183-57dd-446a-af3f-c840b8f24b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:21.262749 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32e1e92-e2e8-4ecd-ad3d-58fb36801b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:21.262939 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68166183-57dd-446a-af3f-c840b8f24b36 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:21.263176 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68166183-57dd-446a-af3f-c840b8f24b36 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:21.263416 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e1e92-e2e8-4ecd-ad3d-58fb36801b18 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:21.263512 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68166183-57dd-446a-af3f-c840b8f24b36 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:21.263692 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e1e92-e2e8-4ecd-ad3d-58fb36801b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:21.263958 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e1e92-e2e8-4ecd-ad3d-58fb36801b18 None admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:21.264684 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32e1e92-e2e8-4ecd-ad3d-58fb36801b18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:21.265016 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32e1e92-e2e8-4ecd-ad3d-58fb36801b18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:21.266386 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32e1e92-e2e8-4ecd-ad3d-58fb36801b18 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:21.274647 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68166183-57dd-446a-af3f-c840b8f24b36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:21.283578 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 328/666] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:21.283578 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32e1e92-e2e8-4ecd-ad3d-58fb36801b18 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:21.283809 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-68166183-57dd-446a-af3f-c840b8f24b36 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:21.283809 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:21.283809 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:21.283809 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:21.283809 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:21.283809 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:21.283809 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:21.283809 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:21.283809 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:21.283809 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:21.283809 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:21.283809 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:21.283809 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:21.283809 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:21.283809 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:21.283809 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:21.284738 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:21.284738 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:21.284738 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:21.284738 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:21.284738 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:21.284738 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:21.284738 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:21.284738 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:21.284738 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:21.284738 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:21.284738 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:21.284738 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:21.284738 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:21.284738 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:21.284738 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:21.284738 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:21.285593 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:21.285593 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:21.285593 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:21.285593 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:21.285593 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:21.285593 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:21.285593 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:21.285593 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:21.285593 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:21.285593 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:21.285593 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:21.285593 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:21.285593 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 339/667] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 27 21:03:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:21.285593 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68166183-57dd-446a-af3f-c840b8f24b36 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:21.299156 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:21.392670 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32e1e92-e2e8-4ecd-ad3d-58fb36801b18 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:21.398532 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68166183-57dd-446a-af3f-c840b8f24b36 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:21.430363 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b70668-2a0d-4b28-8c1c-ffe31016a971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:21.431115 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b70668-2a0d-4b28-8c1c-ffe31016a971 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:21.431403 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b70668-2a0d-4b28-8c1c-ffe31016a971 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:21.431689 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b70668-2a0d-4b28-8c1c-ffe31016a971 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:21.434350 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d932b-cba3-4bef-8eff-1bdf9b76fd59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:21.435152 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d932b-cba3-4bef-8eff-1bdf9b76fd59 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:21.435431 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d932b-cba3-4bef-8eff-1bdf9b76fd59 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:21.435709 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d932b-cba3-4bef-8eff-1bdf9b76fd59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:21.442672 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d932b-cba3-4bef-8eff-1bdf9b76fd59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:21.444144 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d932b-cba3-4bef-8eff-1bdf9b76fd59 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:21.458735 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-386d932b-cba3-4bef-8eff-1bdf9b76fd59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:21.458735 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:21.458735 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:21.458735 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:21.458735 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:21.458735 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:21.458735 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:21.458735 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:21.458735 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:21.458735 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:21.458735 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:21.458735 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:21.458735 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:21.458735 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:21.458735 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:21.459687 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:21.459687 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:21.459687 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:21.459687 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:21.459687 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:21.459687 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:21.459687 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:21.459687 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:21.459687 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:21.459687 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:21.459687 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:21.459687 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:21.459687 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:21.459687 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:21.459687 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:21.459687 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:21.459687 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:21.460584 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:21.460584 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:21.460584 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:21.460584 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:21.460584 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:21.460584 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:21.460584 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:21.460584 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:21.460584 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 340/668] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:21.461522 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d932b-cba3-4bef-8eff-1bdf9b76fd59 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:21.466018 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:21.513685 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b70668-2a0d-4b28-8c1c-ffe31016a971 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:21.515140 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b70668-2a0d-4b28-8c1c-ffe31016a971 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:21.534941 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:21.569652 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d932b-cba3-4bef-8eff-1bdf9b76fd59 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:21.605585 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323815b2-bea0-4dd9-9994-32ba4e3ca5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:21.606253 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323815b2-bea0-4dd9-9994-32ba4e3ca5d7 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:21.606528 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323815b2-bea0-4dd9-9994-32ba4e3ca5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:21.606800 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323815b2-bea0-4dd9-9994-32ba4e3ca5d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:21.613910 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323815b2-bea0-4dd9-9994-32ba4e3ca5d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:21.615377 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323815b2-bea0-4dd9-9994-32ba4e3ca5d7 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:21.618458 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 329/669] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:03:21.619312 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b70668-2a0d-4b28-8c1c-ffe31016a971 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:21.631148 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-323815b2-bea0-4dd9-9994-32ba4e3ca5d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:21.631148 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:21.631148 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:21.631148 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:21.631148 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:21.631148 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:21.631148 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:21.631148 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:21.631148 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:21.631148 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:21.631148 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:21.631148 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:21.631148 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:21.631148 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:21.631148 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:21.632278 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:21.632278 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:21.632278 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:21.632278 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:21.632278 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:21.632278 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:21.632278 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:21.632278 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:21.632278 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:21.632278 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:21.632278 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:21.632278 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:21.632278 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:21.632278 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:21.632278 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:21.632278 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:21.632278 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:21.633230 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:21.633230 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:21.633230 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:21.633230 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:21.633230 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:21.633230 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:21.633230 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:21.633230 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:21.633230 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 341/670] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:21.633230 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323815b2-bea0-4dd9-9994-32ba4e3ca5d7 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:21.693050 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:21.738357 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b70668-2a0d-4b28-8c1c-ffe31016a971 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:21.741443 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323815b2-bea0-4dd9-9994-32ba4e3ca5d7 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:21.768162 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:21.772912 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2790163-94d6-4afa-9fd0-ff7a53450ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:21.773617 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2790163-94d6-4afa-9fd0-ff7a53450ccd None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:21.773899 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2790163-94d6-4afa-9fd0-ff7a53450ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:21.774177 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2790163-94d6-4afa-9fd0-ff7a53450ccd None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:21.774957 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2790163-94d6-4afa-9fd0-ff7a53450ccd None admin] RBAC: Adding query filter params (name=service) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:21.775278 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2790163-94d6-4afa-9fd0-ff7a53450ccd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:21.776399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e883f6-a01e-430e-9ecf-e672cf423f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:21.776742 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2790163-94d6-4afa-9fd0-ff7a53450ccd None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:21.777087 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e883f6-a01e-430e-9ecf-e672cf423f73 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:21.777368 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e883f6-a01e-430e-9ecf-e672cf423f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:21.777642 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e883f6-a01e-430e-9ecf-e672cf423f73 None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:21.778608 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e883f6-a01e-430e-9ecf-e672cf423f73 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:21.778918 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e883f6-a01e-430e-9ecf-e672cf423f73 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:21.780332 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e883f6-a01e-430e-9ecf-e672cf423f73 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:21.785127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 330/671] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 27 21:03:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:21.786490 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5438b6-a23c-406b-a3c7-68813f8be191 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:21.786780 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5438b6-a23c-406b-a3c7-68813f8be191 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:21.787058 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5438b6-a23c-406b-a3c7-68813f8be191 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:21.788295 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 331/672] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:21.789000 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5438b6-a23c-406b-a3c7-68813f8be191 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:21.796094 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 342/673] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:21.797223 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e883f6-a01e-430e-9ecf-e672cf423f73 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:21.900071 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5438b6-a23c-406b-a3c7-68813f8be191 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:21.908566 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e883f6-a01e-430e-9ecf-e672cf423f73 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:21.917167 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:21.935378 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0dd48a-f3e6-43f8-8833-857cf652359c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:21.936188 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0dd48a-f3e6-43f8-8833-857cf652359c None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:21.936475 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0dd48a-f3e6-43f8-8833-857cf652359c None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:21.936772 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0dd48a-f3e6-43f8-8833-857cf652359c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:21.944612 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a379c68c-0b23-45b3-a54f-7e66556c06db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:21.945246 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379c68c-0b23-45b3-a54f-7e66556c06db None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:21.945529 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379c68c-0b23-45b3-a54f-7e66556c06db None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:21.945808 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a379c68c-0b23-45b3-a54f-7e66556c06db None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:21.946606 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a379c68c-0b23-45b3-a54f-7e66556c06db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:21.946916 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a379c68c-0b23-45b3-a54f-7e66556c06db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:21.947728 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0dd48a-f3e6-43f8-8833-857cf652359c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:21.948458 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a379c68c-0b23-45b3-a54f-7e66556c06db None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:21.949313 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0dd48a-f3e6-43f8-8833-857cf652359c None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:21.960161 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 343/674] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:21.961014 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a379c68c-0b23-45b3-a54f-7e66556c06db None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:21.967783 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-fd0dd48a-f3e6-43f8-8833-857cf652359c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:21.967783 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:21.967783 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:21.967783 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:21.967783 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:21.967783 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:21.967783 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:21.967783 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:21.967783 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:21.967783 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:21.967783 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:21.967783 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:21.967783 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:21.967783 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:21.967783 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:21.969793 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:21.969793 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:21.969793 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:21.969793 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:21.969793 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:21.969793 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:21.969793 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:21.969793 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:21.969793 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:21.969793 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:21.969793 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:21.969793 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:21.969793 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:21.969793 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:21.969793 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:21.969793 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:21.969793 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:21.970719 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:21.970719 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:21.970719 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:21.970719 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:21.973239 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 332/675] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:21.974718 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346a952-6c31-4ae3-85f2-83e7c5db0031 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:21.975685 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346a952-6c31-4ae3-85f2-83e7c5db0031 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:21.976119 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346a952-6c31-4ae3-85f2-83e7c5db0031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:21.996322 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-a346a952-6c31-4ae3-85f2-83e7c5db0031 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:21.997574 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-a346a952-6c31-4ae3-85f2-83e7c5db0031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:22.022335 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:22.076654 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a379c68c-0b23-45b3-a54f-7e66556c06db None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:22.078122 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 333/676] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:22.078976 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a346a952-6c31-4ae3-85f2-83e7c5db0031 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:22.115707 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20218beb-d7cd-4a5c-8ee0-e220b9ce897a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:22.116510 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20218beb-d7cd-4a5c-8ee0-e220b9ce897a None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:22.116791 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20218beb-d7cd-4a5c-8ee0-e220b9ce897a None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:22.117147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20218beb-d7cd-4a5c-8ee0-e220b9ce897a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:22.128357 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20218beb-d7cd-4a5c-8ee0-e220b9ce897a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:22.129861 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20218beb-d7cd-4a5c-8ee0-e220b9ce897a None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:22.147211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-20218beb-d7cd-4a5c-8ee0-e220b9ce897a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:22.147211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:22.147211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:22.147211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:22.147211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:22.147211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:22.147211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:22.147211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:22.147211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:22.147211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:22.147211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:22.147211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:22.147211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:22.147211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:22.147211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:22.148142 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:22.148142 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:22.148142 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:22.148142 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:22.148142 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:22.148142 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:22.148142 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:22.148142 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:22.148142 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:22.148142 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:22.148142 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:22.148142 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:22.148142 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:22.148142 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:22.148142 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:22.148142 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:22.148142 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:22.149372 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:22.149372 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:22.149372 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:22.149372 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:22.149372 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 344/677] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:22.149372 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20218beb-d7cd-4a5c-8ee0-e220b9ce897a None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:22.196374 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:22.199404 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a346a952-6c31-4ae3-85f2-83e7c5db0031 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:22.239136 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971b7dd0-bd5a-4463-8cce-a0b3a8bb742b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:22.239847 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b7dd0-bd5a-4463-8cce-a0b3a8bb742b None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:22.240057 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b7dd0-bd5a-4463-8cce-a0b3a8bb742b None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:22.240325 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b7dd0-bd5a-4463-8cce-a0b3a8bb742b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:22.241193 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971b7dd0-bd5a-4463-8cce-a0b3a8bb742b None admin] RBAC: Adding query filter params (role.id=709aef6b85524eb2a8e9986a2bd9525b, scope.project.id=334438c929c44e489e9a3b93e6d37231, user.id=340915e7debe4ce6828bd8a477ed6f16) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:22.241523 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971b7dd0-bd5a-4463-8cce-a0b3a8bb742b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:22.242956 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971b7dd0-bd5a-4463-8cce-a0b3a8bb742b None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:22.249710 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:22.249886 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 334/678] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:03:22 2022] GET /identity/v3/role_assignments?user.id=340915e7debe4ce6828bd8a477ed6f16&scope.project.id=334438c929c44e489e9a3b93e6d37231&role.id=709aef6b85524eb2a8e9986a2bd9525b => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:22.250652 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971b7dd0-bd5a-4463-8cce-a0b3a8bb742b None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:22.265155 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20218beb-d7cd-4a5c-8ee0-e220b9ce897a None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:22.321084 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa5a58e-d858-43a9-a73f-35ebd9401b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:22.321795 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa5a58e-d858-43a9-a73f-35ebd9401b54 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:22.322074 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa5a58e-d858-43a9-a73f-35ebd9401b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:22.322343 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa5a58e-d858-43a9-a73f-35ebd9401b54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:22.323428 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa5a58e-d858-43a9-a73f-35ebd9401b54 None admin] RBAC: Adding query filter params (role.id=709aef6b85524eb2a8e9986a2bd9525b, scope.project.id=334438c929c44e489e9a3b93e6d37231, user.id=26cf962d151b4da7a82c33ef23b342e6) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:22.323715 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa5a58e-d858-43a9-a73f-35ebd9401b54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:22.325460 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa5a58e-d858-43a9-a73f-35ebd9401b54 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:22.336972 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 345/679] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:03:22 2022] GET /identity/v3/role_assignments?user.id=26cf962d151b4da7a82c33ef23b342e6&scope.project.id=334438c929c44e489e9a3b93e6d37231&role.id=709aef6b85524eb2a8e9986a2bd9525b => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:22.336972 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa5a58e-d858-43a9-a73f-35ebd9401b54 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:22.408902 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971b7dd0-bd5a-4463-8cce-a0b3a8bb742b None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:22.467603 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a3facb-2f10-4e21-941a-d83f0a4798cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:22.468194 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3facb-2f10-4e21-941a-d83f0a4798cc None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:22.468474 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3facb-2f10-4e21-941a-d83f0a4798cc None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:22.468799 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3facb-2f10-4e21-941a-d83f0a4798cc None admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:22.469953 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a3facb-2f10-4e21-941a-d83f0a4798cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:22.469953 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a3facb-2f10-4e21-941a-d83f0a4798cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:22.471311 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a3facb-2f10-4e21-941a-d83f0a4798cc None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:22.486764 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa5a58e-d858-43a9-a73f-35ebd9401b54 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:22.486977 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 335/680] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:22.487899 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a3facb-2f10-4e21-941a-d83f0a4798cc None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:22.526541 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:22.534763 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f3f15e-6ce7-4a77-9316-883887ba3561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:22.535580 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f3f15e-6ce7-4a77-9316-883887ba3561 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:22.535859 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f3f15e-6ce7-4a77-9316-883887ba3561 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:22.536142 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f3f15e-6ce7-4a77-9316-883887ba3561 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:22.550887 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f3f15e-6ce7-4a77-9316-883887ba3561 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:22.558850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-94f3f15e-6ce7-4a77-9316-883887ba3561 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:03:22.558850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:22.558850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:22.558850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:22.558850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:22.558850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:22.558850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:22.558850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:22.558850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:22.558850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:22.558850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:22.558850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:22.558850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:22.558850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:22.558850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:22.558850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:22.559830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:22.559830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:22.559830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:22.559830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:22.559830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:22.559830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:22.559830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:22.559830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:22.559830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:22.559830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:22.559830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:22.559830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:22.559830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:22.559830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:22.559830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:22.559830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:22.560710 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:22.560710 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:22.560710 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:22.560710 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:22.560710 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:22.560710 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:22.560710 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:22.560710 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:22.560710 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:22.560710 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:22.560710 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:03:22.560710 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:22.561414 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 346/681] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 27 21:03:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:22.562182 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f3f15e-6ce7-4a77-9316-883887ba3561 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:22.603486 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:22.621469 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a3facb-2f10-4e21-941a-d83f0a4798cc None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:22.688694 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a6da0f-219b-4b8f-8276-fdac00249080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:22.689517 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6da0f-219b-4b8f-8276-fdac00249080 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:22.689800 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6da0f-219b-4b8f-8276-fdac00249080 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:22.690073 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6da0f-219b-4b8f-8276-fdac00249080 None admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:22.690824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a6da0f-219b-4b8f-8276-fdac00249080 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:22.691114 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a6da0f-219b-4b8f-8276-fdac00249080 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:22.692555 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a6da0f-219b-4b8f-8276-fdac00249080 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:22.708080 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 336/682] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:22.708983 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a6da0f-219b-4b8f-8276-fdac00249080 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:22.725813 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f3f15e-6ce7-4a77-9316-883887ba3561 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:22.783178 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb9221-f174-4989-8aff-061b39fd9798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:22.784082 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb9221-f174-4989-8aff-061b39fd9798 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:22.784364 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb9221-f174-4989-8aff-061b39fd9798 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:22.784655 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cb9221-f174-4989-8aff-061b39fd9798 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:22.792469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cb9221-f174-4989-8aff-061b39fd9798 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:22.794292 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cb9221-f174-4989-8aff-061b39fd9798 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:22.807698 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:22.811415 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-29cb9221-f174-4989-8aff-061b39fd9798 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:22.811415 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:22.811415 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:22.811415 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:22.811415 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:22.811415 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:22.811415 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:22.811415 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:22.811415 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:22.811415 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:22.811415 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:22.811415 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:22.811415 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:22.811415 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:22.811415 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:22.812626 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:22.812626 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:22.812626 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:22.812626 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:22.812626 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:22.812626 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:22.812626 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:22.812626 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:22.812626 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:22.812626 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:22.812626 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:22.812626 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:22.812626 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:22.812626 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:22.812626 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:22.812626 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:22.812626 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:22.813571 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:22.813571 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:22.813571 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:22.813571 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:22.813571 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:22.813571 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:22.813571 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:22.813571 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:22.813571 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 347/683] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:22.814080 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb9221-f174-4989-8aff-061b39fd9798 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:22.825116 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a6da0f-219b-4b8f-8276-fdac00249080 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:22.861298 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c45a6-c529-4220-bfb0-a6fa64dc9b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:22.862004 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c45a6-c529-4220-bfb0-a6fa64dc9b21 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:22.862284 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c45a6-c529-4220-bfb0-a6fa64dc9b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:22.862557 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c45a6-c529-4220-bfb0-a6fa64dc9b21 None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:22.863312 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36c45a6-c529-4220-bfb0-a6fa64dc9b21 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:22.863618 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36c45a6-c529-4220-bfb0-a6fa64dc9b21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:22.865027 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36c45a6-c529-4220-bfb0-a6fa64dc9b21 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:22.874588 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 337/684] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:03:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:22.875468 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c45a6-c529-4220-bfb0-a6fa64dc9b21 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:22.908338 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:22.926456 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cb9221-f174-4989-8aff-061b39fd9798 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:22.966156 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35837a00-d3f1-4c2a-a454-31e7a1f25cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:22.966964 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35837a00-d3f1-4c2a-a454-31e7a1f25cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:22.967245 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35837a00-d3f1-4c2a-a454-31e7a1f25cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:22.967520 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35837a00-d3f1-4c2a-a454-31e7a1f25cb9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:22.976154 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35837a00-d3f1-4c2a-a454-31e7a1f25cb9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:22.978848 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35837a00-d3f1-4c2a-a454-31e7a1f25cb9 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:22.981440 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c45a6-c529-4220-bfb0-a6fa64dc9b21 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:22.995314 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-35837a00-d3f1-4c2a-a454-31e7a1f25cb9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 27 21:03:22.995314 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:22.995314 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:22.995314 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:22.995314 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:22.995314 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:22.995314 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:22.995314 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:22.995314 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:22.995314 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:22.995314 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:22.995314 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:22.995314 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:22.995314 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:22.995314 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:22.995314 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:22.997629 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:22.997629 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:22.997629 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:22.997629 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:22.997629 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:22.997629 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:22.997629 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:22.997629 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:22.997629 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:22.997629 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:22.997629 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:22.997629 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:22.997629 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:22.997629 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:22.997629 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:22.997629 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:22.997629 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:22.997629 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:22.999844 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:22.999844 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:22.999844 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:22.999844 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:22.999844 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:22.999844 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:22.999844 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:22.999844 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:22.999844 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 27 21:03:22.999844 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:23.008948 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 348/685] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 27 21:03:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:23.008948 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35837a00-d3f1-4c2a-a454-31e7a1f25cb9 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:23.022188 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10573eb6-4e27-4926-8def-a25b9f087b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:23.022188 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10573eb6-4e27-4926-8def-a25b9f087b57 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:23.022188 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10573eb6-4e27-4926-8def-a25b9f087b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:23.022188 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10573eb6-4e27-4926-8def-a25b9f087b57 None admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:23.023579 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10573eb6-4e27-4926-8def-a25b9f087b57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:23.023579 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10573eb6-4e27-4926-8def-a25b9f087b57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:23.024270 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10573eb6-4e27-4926-8def-a25b9f087b57 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:23.040998 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 338/686] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:23.042067 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10573eb6-4e27-4926-8def-a25b9f087b57 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:23.051745 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:23.118670 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35837a00-d3f1-4c2a-a454-31e7a1f25cb9 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:23.153056 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10573eb6-4e27-4926-8def-a25b9f087b57 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:23.156773 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a3232-2252-46d4-821c-131e89d0b1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:23.157781 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a3232-2252-46d4-821c-131e89d0b1e8 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:23.158229 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a3232-2252-46d4-821c-131e89d0b1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:23.158645 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a3232-2252-46d4-821c-131e89d0b1e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:23.170644 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a3232-2252-46d4-821c-131e89d0b1e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:23.172441 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053a3232-2252-46d4-821c-131e89d0b1e8 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:23.175605 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:23.189349 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a638a3-2729-4103-b7f5-c03c25ed1647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:23.190079 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a638a3-2729-4103-b7f5-c03c25ed1647 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:23.190375 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a638a3-2729-4103-b7f5-c03c25ed1647 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:23.190695 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a638a3-2729-4103-b7f5-c03c25ed1647 None admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:23.191541 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a638a3-2729-4103-b7f5-c03c25ed1647 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:23.191846 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a638a3-2729-4103-b7f5-c03c25ed1647 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:23.193313 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a638a3-2729-4103-b7f5-c03c25ed1647 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:23.195205 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-053a3232-2252-46d4-821c-131e89d0b1e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:23.195205 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:23.195205 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:23.195205 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:23.195205 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:23.195205 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:23.195205 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:23.195205 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:23.195205 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:23.195205 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:23.195205 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:23.195205 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:23.195205 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:23.195205 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:23.195205 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:23.196235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:23.196235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:23.196235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:23.196235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:23.196235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:23.196235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:23.196235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:23.196235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:23.196235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:23.196235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:23.196235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:23.196235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:23.196235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:23.196235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:23.196235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:23.196235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:23.196235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:23.197284 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:23.197284 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:23.197284 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:23.197284 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:23.199267 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 349/687] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:23.200297 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a3232-2252-46d4-821c-131e89d0b1e8 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:23.217705 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 339/688] 10.222.0.13 () {56 vars in 1135 bytes} [Mon Jun 27 21:03:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:23.217705 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a638a3-2729-4103-b7f5-c03c25ed1647 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:23.280359 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:23.319937 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a638a3-2729-4103-b7f5-c03c25ed1647 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:23.320447 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a3232-2252-46d4-821c-131e89d0b1e8 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:23.355441 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f564040-c546-4b96-aee6-7b8aca3c41b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:23.356126 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f564040-c546-4b96-aee6-7b8aca3c41b6 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:23.356406 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f564040-c546-4b96-aee6-7b8aca3c41b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:23.356702 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f564040-c546-4b96-aee6-7b8aca3c41b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:23.357430 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f564040-c546-4b96-aee6-7b8aca3c41b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:23.357733 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f564040-c546-4b96-aee6-7b8aca3c41b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:23.358834 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97576525-49a7-4c3b-8bb1-ca85e2a1bc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:23.359096 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f564040-c546-4b96-aee6-7b8aca3c41b6 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:23.359897 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97576525-49a7-4c3b-8bb1-ca85e2a1bc2d None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:23.360493 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97576525-49a7-4c3b-8bb1-ca85e2a1bc2d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:23.360984 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97576525-49a7-4c3b-8bb1-ca85e2a1bc2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:23.362107 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97576525-49a7-4c3b-8bb1-ca85e2a1bc2d None admin] RBAC: Adding query filter params (role.id=709aef6b85524eb2a8e9986a2bd9525b, scope.project.id=334438c929c44e489e9a3b93e6d37231, user.id=2a4e105858324391b5142d57e34aff9c) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:23.362616 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97576525-49a7-4c3b-8bb1-ca85e2a1bc2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:23.364247 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97576525-49a7-4c3b-8bb1-ca85e2a1bc2d None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:23.371397 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 340/689] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:23.371969 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 350/690] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:03:23 2022] GET /identity/v3/role_assignments?user.id=2a4e105858324391b5142d57e34aff9c&scope.project.id=334438c929c44e489e9a3b93e6d37231&role.id=709aef6b85524eb2a8e9986a2bd9525b => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:23.372290 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f564040-c546-4b96-aee6-7b8aca3c41b6 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:23.377894 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97576525-49a7-4c3b-8bb1-ca85e2a1bc2d None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:23.447496 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:23.498250 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97576525-49a7-4c3b-8bb1-ca85e2a1bc2d None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:23.507312 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f564040-c546-4b96-aee6-7b8aca3c41b6 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:23.534303 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:23.551766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5777fb-48dc-4d78-9094-bfcdc6c61883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:23.552439 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5777fb-48dc-4d78-9094-bfcdc6c61883 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:23.552747 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5777fb-48dc-4d78-9094-bfcdc6c61883 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:23.553220 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5777fb-48dc-4d78-9094-bfcdc6c61883 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:23.581370 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd15ddfc-b304-4c75-8e32-05639bc4b9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:23.583228 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd15ddfc-b304-4c75-8e32-05639bc4b9d2 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:23.583741 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd15ddfc-b304-4c75-8e32-05639bc4b9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:23.584227 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd15ddfc-b304-4c75-8e32-05639bc4b9d2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:23.595377 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd15ddfc-b304-4c75-8e32-05639bc4b9d2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:23.597670 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd15ddfc-b304-4c75-8e32-05639bc4b9d2 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:23.620164 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-cd15ddfc-b304-4c75-8e32-05639bc4b9d2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 21:03:23.620164 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:23.620164 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:23.620164 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:23.620164 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:23.620164 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:23.620164 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:23.620164 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:23.620164 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:23.620164 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:23.620164 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:23.620164 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:23.620164 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:23.620164 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:23.620164 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:23.620164 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:23.621830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:23.621830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:23.621830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:23.621830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:23.621830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:23.621830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:23.621830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:23.621830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:23.621830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:23.621830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:23.621830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:23.621830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:23.621830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:23.621830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:23.621830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:23.621830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:23.621830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:23.621830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:23.622849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:23.622849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:23.622849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:23.622849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:23.622849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:23.622849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:23.622849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:23.622849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:23.622849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 21:03:23.622849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:23.626359 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 351/691] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Jun 27 21:03:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:23.632211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd15ddfc-b304-4c75-8e32-05639bc4b9d2 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:23.656278 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5777fb-48dc-4d78-9094-bfcdc6c61883 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:23.658996 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5777fb-48dc-4d78-9094-bfcdc6c61883 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:23.745991 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd15ddfc-b304-4c75-8e32-05639bc4b9d2 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:23.769418 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:23.785620 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4498dcab-2d68-4425-8db1-e72d91c4f136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:23.786612 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4498dcab-2d68-4425-8db1-e72d91c4f136 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:23.787093 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4498dcab-2d68-4425-8db1-e72d91c4f136 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:23.787563 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4498dcab-2d68-4425-8db1-e72d91c4f136 None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:23.788600 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4498dcab-2d68-4425-8db1-e72d91c4f136 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:23.789146 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4498dcab-2d68-4425-8db1-e72d91c4f136 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:23.790774 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4498dcab-2d68-4425-8db1-e72d91c4f136 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:23.809606 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 352/692] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Jun 27 21:03:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:23.815340 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4498dcab-2d68-4425-8db1-e72d91c4f136 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:23.822346 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 341/693] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 27 21:03:23.833062 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5777fb-48dc-4d78-9094-bfcdc6c61883 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:23.870825 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:23.920888 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4498dcab-2d68-4425-8db1-e72d91c4f136 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:23.956360 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c1d3b5-61ed-4597-b304-d62ee5b703a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:23.957414 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1d3b5-61ed-4597-b304-d62ee5b703a3 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:23.957924 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1d3b5-61ed-4597-b304-d62ee5b703a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:23.958414 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1d3b5-61ed-4597-b304-d62ee5b703a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:24.006152 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5777fb-48dc-4d78-9094-bfcdc6c61883 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:24.008065 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:24.034160 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c1d3b5-61ed-4597-b304-d62ee5b703a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:24.035823 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c1d3b5-61ed-4597-b304-d62ee5b703a3 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:24.077154 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad3982f-d16d-46b9-b8f0-fc44cdb5c7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:24.078707 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3982f-d16d-46b9-b8f0-fc44cdb5c7d2 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:24.079317 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3982f-d16d-46b9-b8f0-fc44cdb5c7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:24.079794 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3982f-d16d-46b9-b8f0-fc44cdb5c7d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:24.120601 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad3982f-d16d-46b9-b8f0-fc44cdb5c7d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:24.123373 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad3982f-d16d-46b9-b8f0-fc44cdb5c7d2 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:24.141928 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 353/694] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 27 21:03:24.147491 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c1d3b5-61ed-4597-b304-d62ee5b703a3 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:24.185046 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-8ad3982f-d16d-46b9-b8f0-fc44cdb5c7d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:24.185046 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:24.185046 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:24.185046 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:24.185046 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:24.185046 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:24.185046 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:24.185046 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:24.185046 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:24.185046 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:24.185046 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:24.185046 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:24.185046 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:24.185046 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:24.185046 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:24.186033 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:24.186033 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:24.186033 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:24.186033 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:24.186033 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:24.186033 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:24.186033 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:24.186033 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:24.186033 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:24.186033 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:24.186033 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:24.186033 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:24.186033 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:24.186033 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:24.186033 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:24.186033 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:24.186033 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:24.186935 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:24.186935 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:24.186935 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:24.186935 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:24.192948 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 342/695] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:24.195796 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad3982f-d16d-46b9-b8f0-fc44cdb5c7d2 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:24.229100 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:24.251379 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c1d3b5-61ed-4597-b304-d62ee5b703a3 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:24.295447 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673bd6ce-6fe3-4d8b-aa5a-ce42a3b1d062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:24.296177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673bd6ce-6fe3-4d8b-aa5a-ce42a3b1d062 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:24.296457 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673bd6ce-6fe3-4d8b-aa5a-ce42a3b1d062 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:24.296730 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673bd6ce-6fe3-4d8b-aa5a-ce42a3b1d062 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:24.312112 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673bd6ce-6fe3-4d8b-aa5a-ce42a3b1d062 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:24.314520 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673bd6ce-6fe3-4d8b-aa5a-ce42a3b1d062 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:24.357519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-673bd6ce-6fe3-4d8b-aa5a-ce42a3b1d062 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:24.357519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:24.357519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:24.357519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:24.357519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:24.357519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:24.357519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:24.357519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:24.357519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:24.357519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:24.357519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:24.357519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:24.357519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:24.357519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:24.357519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:24.359610 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:24.359610 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:24.359610 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:24.359610 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:24.359610 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:24.359610 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:24.359610 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:24.359610 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:24.359610 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:24.359610 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:24.359610 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:24.359610 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:24.359610 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:24.359610 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:24.359610 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:24.359610 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:24.359610 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:24.361732 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:24.361732 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:24.361732 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:24.361732 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:24.368186 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 354/696] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:24.375236 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673bd6ce-6fe3-4d8b-aa5a-ce42a3b1d062 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:24.395686 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:24.449761 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad3982f-d16d-46b9-b8f0-fc44cdb5c7d2 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:24.517036 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673bd6ce-6fe3-4d8b-aa5a-ce42a3b1d062 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:24.527360 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a26efd-ca0f-47b9-bf2f-440655fa1890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:24.528480 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a26efd-ca0f-47b9-bf2f-440655fa1890 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:24.529022 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a26efd-ca0f-47b9-bf2f-440655fa1890 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:24.529591 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a26efd-ca0f-47b9-bf2f-440655fa1890 None admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:24.530675 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a26efd-ca0f-47b9-bf2f-440655fa1890 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:24.531246 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a26efd-ca0f-47b9-bf2f-440655fa1890 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:24.533096 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a26efd-ca0f-47b9-bf2f-440655fa1890 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:24.545367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:24.550589 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948bde0-72d1-4950-8f33-c011a5d53201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:24.551263 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948bde0-72d1-4950-8f33-c011a5d53201 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:24.551549 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948bde0-72d1-4950-8f33-c011a5d53201 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:24.551848 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948bde0-72d1-4950-8f33-c011a5d53201 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:24.552789 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7948bde0-72d1-4950-8f33-c011a5d53201 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:24.553196 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7948bde0-72d1-4950-8f33-c011a5d53201 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:24.554858 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7948bde0-72d1-4950-8f33-c011a5d53201 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:24.561545 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 343/697] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:24.567380 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a26efd-ca0f-47b9-bf2f-440655fa1890 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:24.569443 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 355/698] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:24.576128 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948bde0-72d1-4950-8f33-c011a5d53201 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:24.684340 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948bde0-72d1-4950-8f33-c011a5d53201 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:24.722697 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f809ffb-9d10-4046-be13-a217b2a67714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:24.723511 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f809ffb-9d10-4046-be13-a217b2a67714 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:24.723948 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f809ffb-9d10-4046-be13-a217b2a67714 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:24.724233 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f809ffb-9d10-4046-be13-a217b2a67714 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:24.732359 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f809ffb-9d10-4046-be13-a217b2a67714 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:24.733993 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f809ffb-9d10-4046-be13-a217b2a67714 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:24.749123 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-7f809ffb-9d10-4046-be13-a217b2a67714 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:24.749123 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:24.749123 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:24.749123 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:24.749123 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:24.749123 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:24.749123 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:24.749123 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:24.749123 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:24.749123 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:24.749123 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:24.749123 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:24.749123 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:24.749123 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:24.749123 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:24.750047 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:24.750047 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:24.750047 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:24.750047 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:24.750047 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:24.750047 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:24.750047 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:24.750047 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:24.750047 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:24.750047 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:24.750047 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:24.750047 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:24.750047 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:24.750047 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:24.750047 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:24.750047 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:24.750047 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:24.750959 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:24.750959 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:24.750959 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:24.750959 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:24.750959 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:24.750959 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:24.750959 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:24.750959 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:24.750959 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 356/699] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:24.756302 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f809ffb-9d10-4046-be13-a217b2a67714 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:24.763454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a26efd-ca0f-47b9-bf2f-440655fa1890 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:24.821416 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:24.831899 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395592ce-9921-4f3a-a833-b80558019139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:24.832855 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395592ce-9921-4f3a-a833-b80558019139 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:24.833162 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395592ce-9921-4f3a-a833-b80558019139 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:24.833433 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395592ce-9921-4f3a-a833-b80558019139 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:24.837709 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:24.845585 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395592ce-9921-4f3a-a833-b80558019139 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:24.847112 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395592ce-9921-4f3a-a833-b80558019139 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:24.864820 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-395592ce-9921-4f3a-a833-b80558019139 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:24.864820 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:24.864820 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:24.864820 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:24.864820 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:24.864820 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:24.864820 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:24.864820 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:24.864820 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:24.864820 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:24.864820 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:24.864820 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:24.864820 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:24.864820 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:24.864820 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:24.865824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:24.865824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:24.865824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:24.865824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:24.865824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:24.865824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:24.865824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:24.865824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:24.865824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:24.865824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:24.865824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:24.865824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:24.865824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:24.865824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:24.865824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:24.865824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:24.865824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:24.866793 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:24.866793 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:24.866793 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:24.866793 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:24.866793 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:24.866793 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:24.866793 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:24.866793 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:24.866793 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 344/700] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:24.867756 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507a502-24ee-4eee-a0a7-a86bfd9e656b None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:24.868888 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507a502-24ee-4eee-a0a7-a86bfd9e656b None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:24.868888 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507a502-24ee-4eee-a0a7-a86bfd9e656b None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:24.874109 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 345/701] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:24.874944 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7507a502-24ee-4eee-a0a7-a86bfd9e656b None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:24.880614 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f809ffb-9d10-4046-be13-a217b2a67714 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:24.921006 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd780707-519c-4b74-8186-bfb08e88f36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:24.921785 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd780707-519c-4b74-8186-bfb08e88f36c None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:24.922075 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd780707-519c-4b74-8186-bfb08e88f36c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:24.922346 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd780707-519c-4b74-8186-bfb08e88f36c None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:24.923225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd780707-519c-4b74-8186-bfb08e88f36c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:24.923602 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd780707-519c-4b74-8186-bfb08e88f36c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:24.925347 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd780707-519c-4b74-8186-bfb08e88f36c None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:24.942080 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 357/702] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:24.949067 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b1da5-2c0e-4b01-829a-ca65b51f2c4e None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:24.949623 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b1da5-2c0e-4b01-829a-ca65b51f2c4e None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:24.949915 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b1da5-2c0e-4b01-829a-ca65b51f2c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:24.968454 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-4c6b1da5-2c0e-4b01-829a-ca65b51f2c4e None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:24.969771 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-4c6b1da5-2c0e-4b01-829a-ca65b51f2c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:25.007297 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7507a502-24ee-4eee-a0a7-a86bfd9e656b None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:25.061568 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 358/703] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:25.062854 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b1da5-2c0e-4b01-829a-ca65b51f2c4e None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:25.103891 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016918d-f978-44f3-9f8b-e1859934071a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:25.104615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016918d-f978-44f3-9f8b-e1859934071a None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:25.105124 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016918d-f978-44f3-9f8b-e1859934071a None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:25.105572 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016918d-f978-44f3-9f8b-e1859934071a None admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:25.106656 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4016918d-f978-44f3-9f8b-e1859934071a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:25.107000 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4016918d-f978-44f3-9f8b-e1859934071a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:25.109312 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4016918d-f978-44f3-9f8b-e1859934071a None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:25.125621 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 346/704] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:25.126877 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:25.127971 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016918d-f978-44f3-9f8b-e1859934071a None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:25.190417 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:25.190998 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b1da5-2c0e-4b01-829a-ca65b51f2c4e None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:25.228831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2044c1f9-d247-4886-bfb2-f36315fd0392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:25.230086 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044c1f9-d247-4886-bfb2-f36315fd0392 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:25.230362 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044c1f9-d247-4886-bfb2-f36315fd0392 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:25.230660 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2044c1f9-d247-4886-bfb2-f36315fd0392 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:25.231474 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2044c1f9-d247-4886-bfb2-f36315fd0392 None admin] RBAC: Adding query filter params (role.id=d336dd7b746c4488b33289c00302e4d5, scope.project.id=334438c929c44e489e9a3b93e6d37231, user.id=1366958034d540b4ba2feb421a6afabc) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:25.231786 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2044c1f9-d247-4886-bfb2-f36315fd0392 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:25.233255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2044c1f9-d247-4886-bfb2-f36315fd0392 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:25.244958 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 359/705] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:03:25 2022] GET /identity/v3/role_assignments?user.id=1366958034d540b4ba2feb421a6afabc&scope.project.id=334438c929c44e489e9a3b93e6d37231&role.id=d336dd7b746c4488b33289c00302e4d5 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:25.244958 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4119df-5319-46a5-9fd2-40eb4ac2726c None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:25.245219 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016918d-f978-44f3-9f8b-e1859934071a None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:25.245324 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4119df-5319-46a5-9fd2-40eb4ac2726c None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:25.245324 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4119df-5319-46a5-9fd2-40eb4ac2726c None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:25.245324 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 360/706] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:25.245324 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4119df-5319-46a5-9fd2-40eb4ac2726c None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:25.286067 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dee284-bbfd-43a9-a704-4244f4d40a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:25.286901 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dee284-bbfd-43a9-a704-4244f4d40a50 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:25.287179 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dee284-bbfd-43a9-a704-4244f4d40a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:25.287552 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dee284-bbfd-43a9-a704-4244f4d40a50 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:25.288236 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dee284-bbfd-43a9-a704-4244f4d40a50 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:25.289632 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dee284-bbfd-43a9-a704-4244f4d40a50 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:25.305584 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-f9dee284-bbfd-43a9-a704-4244f4d40a50 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 21:03:25.305584 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:25.305584 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:03:25.305584 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:03:25.305584 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:03:25.305584 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:03:25.305584 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:03:25.305584 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:25.305584 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 21:03:25.305584 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:25.305584 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:03:25.305584 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:25.305584 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:25.305584 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:25.305584 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:25.305584 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:25.307582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:25.307582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:25.307582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:25.307582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:25.307582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:25.307582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:25.307582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:25.307582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:03:25.307582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:03:25.307582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:03:25.307582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:03:25.307582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:25.307582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:25.307582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:25.307582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:25.307582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:25.308520 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:25.308520 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:25.308520 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:25.308520 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:25.308520 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:25.308520 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:25.308520 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:25.308520 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:25.308520 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:25.308520 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:25.308520 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:25.308520 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:03:25.308520 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:25.308520 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 21:03:25.308520 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:25.312854 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 347/707] 10.222.0.13 () {56 vars in 1129 bytes} [Mon Jun 27 21:03:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:25.314430 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740b66f-ad60-4d24-9231-76803c4ac67f None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:25.314721 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740b66f-ad60-4d24-9231-76803c4ac67f None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:25.315000 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740b66f-ad60-4d24-9231-76803c4ac67f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:25.336336 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-7740b66f-ad60-4d24-9231-76803c4ac67f None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:25.337649 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-7740b66f-ad60-4d24-9231-76803c4ac67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:25.410262 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4119df-5319-46a5-9fd2-40eb4ac2726c None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:25.414445 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:25.452359 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 348/708] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:25.452359 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7740b66f-ad60-4d24-9231-76803c4ac67f None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:25.466706 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a094c11-8712-4b20-b271-882aafe11e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:25.467514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a094c11-8712-4b20-b271-882aafe11e8d None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:25.467797 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a094c11-8712-4b20-b271-882aafe11e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:25.468074 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a094c11-8712-4b20-b271-882aafe11e8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:25.468952 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a094c11-8712-4b20-b271-882aafe11e8d None admin] RBAC: Adding query filter params (role.id=709aef6b85524eb2a8e9986a2bd9525b, scope.project.id=334438c929c44e489e9a3b93e6d37231, user.id=8bed6b231ee647d8a350da055d8e59d7) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:25.469267 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a094c11-8712-4b20-b271-882aafe11e8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:25.471021 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a094c11-8712-4b20-b271-882aafe11e8d None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:25.485476 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 361/709] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:03:25 2022] GET /identity/v3/role_assignments?user.id=8bed6b231ee647d8a350da055d8e59d7&scope.project.id=334438c929c44e489e9a3b93e6d37231&role.id=709aef6b85524eb2a8e9986a2bd9525b => generated 608 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:25.486401 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a094c11-8712-4b20-b271-882aafe11e8d None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:25.552700 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:25.637373 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7740b66f-ad60-4d24-9231-76803c4ac67f None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:25.668274 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a094c11-8712-4b20-b271-882aafe11e8d None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:25.726030 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf75ec5-b600-4798-9bde-842c3be5d9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:25.727583 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf75ec5-b600-4798-9bde-842c3be5d9ef None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:25.729998 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf75ec5-b600-4798-9bde-842c3be5d9ef None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:25.730290 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf75ec5-b600-4798-9bde-842c3be5d9ef None admin] PATH_INFO: `/v3/services` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:25.731148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf75ec5-b600-4798-9bde-842c3be5d9ef None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:25.731448 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf75ec5-b600-4798-9bde-842c3be5d9ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:25.732868 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf75ec5-b600-4798-9bde-842c3be5d9ef None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:25.739656 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6124e229-7da1-469e-9249-f782ec4742de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:25.740485 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124e229-7da1-469e-9249-f782ec4742de None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:25.740767 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124e229-7da1-469e-9249-f782ec4742de None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:25.743008 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6124e229-7da1-469e-9249-f782ec4742de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:25.749040 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 349/710] 10.222.0.13 () {56 vars in 1124 bytes} [Mon Jun 27 21:03:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:25.751635 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf75ec5-b600-4798-9bde-842c3be5d9ef None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:25.759009 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6124e229-7da1-469e-9249-f782ec4742de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:25.761811 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6124e229-7da1-469e-9249-f782ec4742de None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:25.788054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-6124e229-7da1-469e-9249-f782ec4742de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:25.788054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:25.788054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:25.788054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:25.788054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:25.788054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:25.788054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:25.788054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:25.788054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:25.788054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:25.788054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:25.788054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:25.788054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:25.788054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:25.788054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:25.789089 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:25.789089 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:25.789089 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:25.789089 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:25.789089 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:25.789089 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:25.789089 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:25.789089 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:25.789089 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:25.789089 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:25.789089 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:25.789089 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:25.789089 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:25.789089 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:25.789089 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:25.789089 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:25.789089 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:25.789979 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:25.789979 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:25.789979 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:25.789979 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:25.790920 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 362/711] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:25.797676 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6124e229-7da1-469e-9249-f782ec4742de None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:25.827679 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:25.885499 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf75ec5-b600-4798-9bde-842c3be5d9ef None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:25.902734 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:25.928532 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6124e229-7da1-469e-9249-f782ec4742de None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:25.929932 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50635e5f-db93-4c06-b361-eefbcc6d1b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:25.931717 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50635e5f-db93-4c06-b361-eefbcc6d1b55 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:25.932214 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50635e5f-db93-4c06-b361-eefbcc6d1b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:25.933462 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50635e5f-db93-4c06-b361-eefbcc6d1b55 None admin] PATH_INFO: `/v3/services` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:25.934372 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50635e5f-db93-4c06-b361-eefbcc6d1b55 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:25.934854 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50635e5f-db93-4c06-b361-eefbcc6d1b55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:25.936408 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50635e5f-db93-4c06-b361-eefbcc6d1b55 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:25.944501 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 350/712] 10.222.0.13 () {56 vars in 1124 bytes} [Mon Jun 27 21:03:25 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:25.969713 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d9e0c5-80de-463a-9ff5-46ab19aae6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:25.970801 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9e0c5-80de-463a-9ff5-46ab19aae6cd None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:25.971298 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9e0c5-80de-463a-9ff5-46ab19aae6cd None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:25.971764 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9e0c5-80de-463a-9ff5-46ab19aae6cd None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:25.972722 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d9e0c5-80de-463a-9ff5-46ab19aae6cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:25.973292 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d9e0c5-80de-463a-9ff5-46ab19aae6cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:25.974929 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d9e0c5-80de-463a-9ff5-46ab19aae6cd None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:25.989882 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 363/713] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:25.994724 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0687eb0e-8f57-461f-a3e3-3c531a7e10ac None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:25.995085 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0687eb0e-8f57-461f-a3e3-3c531a7e10ac None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:25.995983 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0687eb0e-8f57-461f-a3e3-3c531a7e10ac None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:25.997696 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 351/714] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:26.002439 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d9e0c5-80de-463a-9ff5-46ab19aae6cd None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:26.003168 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880917f7-d4e3-44a5-b01b-3f034b578c92 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:26.003538 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880917f7-d4e3-44a5-b01b-3f034b578c92 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:26.003821 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880917f7-d4e3-44a5-b01b-3f034b578c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:26.031030 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-880917f7-d4e3-44a5-b01b-3f034b578c92 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:26.032242 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-880917f7-d4e3-44a5-b01b-3f034b578c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:26.114730 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 352/715] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:03:26.119013 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:26.141229 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880917f7-d4e3-44a5-b01b-3f034b578c92 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:26.150873 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d9e0c5-80de-463a-9ff5-46ab19aae6cd None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:26.187210 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8104c82-5b46-41eb-8678-6a81739d18f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:26.187809 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8104c82-5b46-41eb-8678-6a81739d18f9 None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:26.188091 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8104c82-5b46-41eb-8678-6a81739d18f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:26.188357 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8104c82-5b46-41eb-8678-6a81739d18f9 None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:26.189101 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8104c82-5b46-41eb-8678-6a81739d18f9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:26.190545 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8104c82-5b46-41eb-8678-6a81739d18f9 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:26.215103 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 364/716] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Jun 27 21:03:26 2022] POST /identity/v3/users => generated 254 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:03:26.259607 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880917f7-d4e3-44a5-b01b-3f034b578c92 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:26.271832 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:26.303327 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2969b5a-76fc-4197-977d-7d01650123d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:26.304159 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2969b5a-76fc-4197-977d-7d01650123d4 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:26.304447 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2969b5a-76fc-4197-977d-7d01650123d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:26.304724 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2969b5a-76fc-4197-977d-7d01650123d4 None admin] PATH_INFO: `/v3/services/network` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:26.305501 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2969b5a-76fc-4197-977d-7d01650123d4 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:26.306926 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2969b5a-76fc-4197-977d-7d01650123d4 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:26.320350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-e2969b5a-76fc-4197-977d-7d01650123d4 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 21:03:26.320350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:26.320350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:03:26.320350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:03:26.320350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:03:26.320350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:03:26.320350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:03:26.320350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:26.320350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 21:03:26.320350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:26.320350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:03:26.320350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:26.320350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:26.320350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:26.320350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:26.320350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:26.320350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:26.321449 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:26.321449 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:26.321449 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:26.321449 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:26.321449 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:26.321449 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:26.321449 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:03:26.321449 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:03:26.321449 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:03:26.321449 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:03:26.321449 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:26.321449 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:26.321449 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:26.321449 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:26.321449 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:26.321449 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:26.322271 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:26.322271 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:26.322271 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:26.322271 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:26.322271 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:26.322271 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:26.322271 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:26.322271 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:26.322271 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:26.322271 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:26.322271 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:03:26.322271 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:26.322271 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 21:03:26.322271 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:26.326997 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 353/717] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:26 2022] GET /identity/v3/services/network => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:26.328478 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8104c82-5b46-41eb-8678-6a81739d18f9 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:26.444964 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8104c82-5b46-41eb-8678-6a81739d18f9 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:26.483068 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc2596-fcfd-43a2-a08a-ef92db3c9227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:26.483745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc2596-fcfd-43a2-a08a-ef92db3c9227 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:26.484028 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc2596-fcfd-43a2-a08a-ef92db3c9227 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:26.484305 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc2596-fcfd-43a2-a08a-ef92db3c9227 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:26.485096 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc2596-fcfd-43a2-a08a-ef92db3c9227 None admin] RBAC: Adding query filter params (name=network) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:26.485556 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc2596-fcfd-43a2-a08a-ef92db3c9227 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:26.487028 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc2596-fcfd-43a2-a08a-ef92db3c9227 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:26.489209 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:26.497324 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 365/718] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Jun 27 21:03:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:26.498778 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2969b5a-76fc-4197-977d-7d01650123d4 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:26.610921 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2969b5a-76fc-4197-977d-7d01650123d4 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:26.647684 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14723ef6-f99c-457a-9001-f8ab688100f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:26.648774 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14723ef6-f99c-457a-9001-f8ab688100f0 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:26.649404 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14723ef6-f99c-457a-9001-f8ab688100f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:26.649895 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14723ef6-f99c-457a-9001-f8ab688100f0 None admin] PATH_INFO: `/v3/services` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:26.650804 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14723ef6-f99c-457a-9001-f8ab688100f0 None admin] RBAC: Adding query filter params (type=network) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:26.651437 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14723ef6-f99c-457a-9001-f8ab688100f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:26.653039 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14723ef6-f99c-457a-9001-f8ab688100f0 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:26.658382 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 354/719] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Jun 27 21:03:26 2022] GET /identity/v3/services?type=network => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:27.534306 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c819a3d-68ed-41f6-be5d-86a7f8f4b079 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:27.534845 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c819a3d-68ed-41f6-be5d-86a7f8f4b079 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:27.534845 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c819a3d-68ed-41f6-be5d-86a7f8f4b079 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:27.536452 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 366/720] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:27.542124 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715d1ce-8067-4319-9bb8-848d5845ab25 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:27.542576 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715d1ce-8067-4319-9bb8-848d5845ab25 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:27.542755 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715d1ce-8067-4319-9bb8-848d5845ab25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:27.566064 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-5715d1ce-8067-4319-9bb8-848d5845ab25 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:27.567422 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-5715d1ce-8067-4319-9bb8-848d5845ab25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:27.614495 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:27.667554 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 355/721] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:03:27.704354 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c819a3d-68ed-41f6-be5d-86a7f8f4b079 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:27.819129 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c819a3d-68ed-41f6-be5d-86a7f8f4b079 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:27.858204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee43f8-07db-4a7e-8d45-d135671bfda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:27.859017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee43f8-07db-4a7e-8d45-d135671bfda8 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:27.859292 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee43f8-07db-4a7e-8d45-d135671bfda8 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:27.859558 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee43f8-07db-4a7e-8d45-d135671bfda8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:27.873880 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee43f8-07db-4a7e-8d45-d135671bfda8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:27.881458 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-ecee43f8-07db-4a7e-8d45-d135671bfda8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:03:27.881458 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:27.881458 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:27.881458 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:27.881458 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:27.881458 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:27.881458 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:27.881458 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:27.881458 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:27.881458 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:27.881458 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:27.881458 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:27.881458 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:27.881458 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:27.881458 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:27.881458 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:27.882358 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:27.882358 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:27.882358 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:27.882358 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:27.882358 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:27.882358 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:27.882358 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:27.882358 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:27.882358 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:27.882358 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:27.882358 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:27.882358 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:27.882358 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:27.882358 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:27.882358 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:27.882358 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:27.883249 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:27.883249 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:27.883249 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:27.883249 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:27.883249 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:27.883249 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:27.883249 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:27.883249 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:27.883249 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:27.883249 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:27.883249 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:03:27.883249 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:27.883249 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 367/722] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 27 21:03:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:27.890644 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715d1ce-8067-4319-9bb8-848d5845ab25 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:28.042153 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715d1ce-8067-4319-9bb8-848d5845ab25 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:28.086133 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c5f77-d04e-44f1-8cd5-2cc579c205a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:28.086774 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c5f77-d04e-44f1-8cd5-2cc579c205a4 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:28.087054 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c5f77-d04e-44f1-8cd5-2cc579c205a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:28.087330 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c5f77-d04e-44f1-8cd5-2cc579c205a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:28.088131 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2c5f77-d04e-44f1-8cd5-2cc579c205a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:28.088442 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2c5f77-d04e-44f1-8cd5-2cc579c205a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:28.098384 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2c5f77-d04e-44f1-8cd5-2cc579c205a4 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:28.108568 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 356/723] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:03:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:28.114288 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee43f8-07db-4a7e-8d45-d135671bfda8 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:28.228827 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee43f8-07db-4a7e-8d45-d135671bfda8 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:28.253420 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:28.264789 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6338d1-7ece-42c2-9eee-6bfe45ceb983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:28.265680 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6338d1-7ece-42c2-9eee-6bfe45ceb983 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:28.265958 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6338d1-7ece-42c2-9eee-6bfe45ceb983 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:28.266276 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6338d1-7ece-42c2-9eee-6bfe45ceb983 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:28.277637 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6338d1-7ece-42c2-9eee-6bfe45ceb983 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:28.279130 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6338d1-7ece-42c2-9eee-6bfe45ceb983 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:28.297870 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-8a6338d1-7ece-42c2-9eee-6bfe45ceb983 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:28.297870 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:28.297870 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:28.297870 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:28.297870 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:28.297870 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:28.297870 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:28.297870 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:28.297870 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:28.297870 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:28.297870 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:28.297870 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:28.297870 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:28.297870 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:28.297870 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:28.298824 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:28.298824 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:28.298824 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:28.298824 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:28.298824 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:28.298824 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:28.298824 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:28.298824 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:28.298824 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:28.298824 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:28.298824 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:28.298824 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:28.298824 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:28.298824 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:28.298824 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:28.298824 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:28.298824 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:28.299775 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:28.299775 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:28.299775 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:28.299775 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:28.303968 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 368/724] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:28.306066 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c5f77-d04e-44f1-8cd5-2cc579c205a4 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:28.361390 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:28.468140 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c5f77-d04e-44f1-8cd5-2cc579c205a4 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:28.507715 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e1c01b-3d18-44fd-9e23-8249935b7c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:28.508403 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e1c01b-3d18-44fd-9e23-8249935b7c4a None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:28.508685 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e1c01b-3d18-44fd-9e23-8249935b7c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:28.508986 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e1c01b-3d18-44fd-9e23-8249935b7c4a None admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:28.509725 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e1c01b-3d18-44fd-9e23-8249935b7c4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:28.510035 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e1c01b-3d18-44fd-9e23-8249935b7c4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:28.511457 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e1c01b-3d18-44fd-9e23-8249935b7c4a None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:28.534335 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 357/725] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:28.542069 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6338d1-7ece-42c2-9eee-6bfe45ceb983 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:28.620462 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0341c-3f66-4011-84f7-5c82eebad624 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:28.621029 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0341c-3f66-4011-84f7-5c82eebad624 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:28.621029 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0341c-3f66-4011-84f7-5c82eebad624 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:28.623604 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 358/726] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:03:28.627845 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d42ee4-ce15-4a31-b9c9-22937647bf15 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:28.628162 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d42ee4-ce15-4a31-b9c9-22937647bf15 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:28.628416 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d42ee4-ce15-4a31-b9c9-22937647bf15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:28.653107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-d8d42ee4-ce15-4a31-b9c9-22937647bf15 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:28.655710 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-d8d42ee4-ce15-4a31-b9c9-22937647bf15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:28.698647 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6338d1-7ece-42c2-9eee-6bfe45ceb983 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:28.717314 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:28.751392 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554376f-3ef6-4620-9961-8d18a881245d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:28.752758 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554376f-3ef6-4620-9961-8d18a881245d None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:28.753262 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554376f-3ef6-4620-9961-8d18a881245d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:28.753747 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554376f-3ef6-4620-9961-8d18a881245d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:28.757086 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 359/727] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 27 21:03:28.758949 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe4d3c-ee1a-4543-8cba-b85988e6d971 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:28.759840 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe4d3c-ee1a-4543-8cba-b85988e6d971 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:28.760124 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe4d3c-ee1a-4543-8cba-b85988e6d971 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:28.762107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 360/728] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:28.775865 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d554376f-3ef6-4620-9961-8d18a881245d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:28.776132 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfc7e6-97b0-45cf-8435-e2ffe5d0db3e None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:28.776316 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfc7e6-97b0-45cf-8435-e2ffe5d0db3e None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:28.776599 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfc7e6-97b0-45cf-8435-e2ffe5d0db3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:28.779183 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d554376f-3ef6-4620-9961-8d18a881245d None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:28.801205 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-92cfc7e6-97b0-45cf-8435-e2ffe5d0db3e None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:28.802412 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-92cfc7e6-97b0-45cf-8435-e2ffe5d0db3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:28.810766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-d554376f-3ef6-4620-9961-8d18a881245d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 21:03:28.810766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:28.810766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:28.810766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:28.810766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:28.810766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:28.810766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:28.810766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:28.810766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:28.810766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:28.810766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:28.810766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:28.810766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:28.810766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:28.810766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:28.810766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:28.811745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:28.811745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:28.811745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:28.811745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:28.811745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:28.811745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:28.811745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:28.811745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:28.811745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:28.811745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:28.811745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:28.811745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:28.811745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:28.811745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:28.811745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:28.811745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:28.811745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:28.811745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:28.812699 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:28.812699 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:28.812699 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:28.812699 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:28.812699 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:28.812699 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:28.812699 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:28.812699 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:28.812699 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 21:03:28.812699 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:28.814795 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 369/729] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Jun 27 21:03:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:28.817158 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554376f-3ef6-4620-9961-8d18a881245d None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:28.818913 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:28.883554 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 361/730] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:28.889915 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cfc7e6-97b0-45cf-8435-e2ffe5d0db3e None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:28.933484 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554376f-3ef6-4620-9961-8d18a881245d None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:28.971164 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f46b4-2eaf-4afb-a4ab-2beb94cf141b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:28.972071 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f46b4-2eaf-4afb-a4ab-2beb94cf141b None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:28.972373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f46b4-2eaf-4afb-a4ab-2beb94cf141b None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:28.972716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f46b4-2eaf-4afb-a4ab-2beb94cf141b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:28.984985 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5f46b4-2eaf-4afb-a4ab-2beb94cf141b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:28.993038 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-ed5f46b4-2eaf-4afb-a4ab-2beb94cf141b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:03:28.993038 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:28.993038 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:28.993038 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:28.993038 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:28.993038 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:28.993038 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:28.993038 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:28.993038 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:28.993038 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:28.993038 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:28.993038 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:28.993038 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:28.993038 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:28.993038 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:28.993038 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:28.995221 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:28.995221 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:28.995221 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:28.995221 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:28.995221 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:28.995221 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:28.995221 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:28.995221 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:28.995221 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:28.995221 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:28.995221 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:28.995221 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:28.995221 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:28.995221 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:28.995221 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:28.995221 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:28.997187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:28.997187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:28.997187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:28.997187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:28.997187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:28.997187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:28.997187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:28.997187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:28.997187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:28.997187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:28.997187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:03:28.997187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:29.005191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 370/731] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 27 21:03:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:29.005191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f46b4-2eaf-4afb-a4ab-2beb94cf141b None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:29.020224 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cfc7e6-97b0-45cf-8435-e2ffe5d0db3e None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:29.035270 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:29.061050 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b5ef3e-8063-49df-bd83-20e7e0590429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:29.061775 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5ef3e-8063-49df-bd83-20e7e0590429 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:29.062068 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5ef3e-8063-49df-bd83-20e7e0590429 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:29.062346 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5ef3e-8063-49df-bd83-20e7e0590429 None admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:29.063212 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b5ef3e-8063-49df-bd83-20e7e0590429 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:29.063625 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b5ef3e-8063-49df-bd83-20e7e0590429 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:29.065218 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b5ef3e-8063-49df-bd83-20e7e0590429 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:29.085027 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 362/732] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Jun 27 21:03:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:29.086025 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b5ef3e-8063-49df-bd83-20e7e0590429 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:29.095395 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:29.116570 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f46b4-2eaf-4afb-a4ab-2beb94cf141b None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:29.155017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbcda7c-f3a7-4b27-bfe9-7f03c3ed9179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:29.155327 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbcda7c-f3a7-4b27-bfe9-7f03c3ed9179 None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:29.155494 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbcda7c-f3a7-4b27-bfe9-7f03c3ed9179 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:29.155745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbcda7c-f3a7-4b27-bfe9-7f03c3ed9179 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:29.156557 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbcda7c-f3a7-4b27-bfe9-7f03c3ed9179 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:29.158116 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbcda7c-f3a7-4b27-bfe9-7f03c3ed9179 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:29.177294 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 371/733] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Jun 27 21:03:28 2022] POST /identity/v3/services => generated 251 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:03:29.178317 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbcda7c-f3a7-4b27-bfe9-7f03c3ed9179 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:29.229976 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b5ef3e-8063-49df-bd83-20e7e0590429 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:29.299367 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214faf29-15bf-4bd0-8b13-829fed77c6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:29.300057 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214faf29-15bf-4bd0-8b13-829fed77c6e1 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:29.300339 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214faf29-15bf-4bd0-8b13-829fed77c6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:29.300615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214faf29-15bf-4bd0-8b13-829fed77c6e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:29.301412 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214faf29-15bf-4bd0-8b13-829fed77c6e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:29.301718 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214faf29-15bf-4bd0-8b13-829fed77c6e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:29.303233 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214faf29-15bf-4bd0-8b13-829fed77c6e1 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:29.318669 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 363/734] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:03:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:29.320262 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff20460-8277-4b20-bac8-f5bd9498ae89 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:29.320554 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff20460-8277-4b20-bac8-f5bd9498ae89 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:29.320828 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff20460-8277-4b20-bac8-f5bd9498ae89 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:29.322544 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 364/735] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:29.323582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff20460-8277-4b20-bac8-f5bd9498ae89 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:29.345724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:29.359893 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbcda7c-f3a7-4b27-bfe9-7f03c3ed9179 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:29.404145 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d852608-853b-496e-be95-219f3ae3b491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:29.405119 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d852608-853b-496e-be95-219f3ae3b491 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:29.405326 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d852608-853b-496e-be95-219f3ae3b491 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:29.405654 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d852608-853b-496e-be95-219f3ae3b491 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:29.410985 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:29.421798 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d852608-853b-496e-be95-219f3ae3b491 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:29.423285 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d852608-853b-496e-be95-219f3ae3b491 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:29.441297 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-4d852608-853b-496e-be95-219f3ae3b491 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:29.441297 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:29.441297 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:29.441297 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:29.441297 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:29.441297 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:29.441297 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:29.441297 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:29.441297 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:29.441297 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:29.441297 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:29.441297 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:29.441297 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:29.441297 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:29.441297 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:29.442299 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:29.442299 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:29.442299 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:29.442299 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:29.442299 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:29.442299 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:29.442299 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:29.442299 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:29.442299 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:29.442299 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:29.442299 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:29.442299 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:29.442299 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:29.442299 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:29.442299 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:29.442299 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:29.442299 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:29.443293 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:29.443293 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:29.443293 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:29.443293 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:29.443293 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 372/736] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:29.443793 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4e31c-afac-4bb7-8c44-41ad3c39ae3b None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:29.444021 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4e31c-afac-4bb7-8c44-41ad3c39ae3b None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:29.444294 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4e31c-afac-4bb7-8c44-41ad3c39ae3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:29.458279 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff20460-8277-4b20-bac8-f5bd9498ae89 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:29.461554 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-3fb4e31c-afac-4bb7-8c44-41ad3c39ae3b None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:29.462767 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-3fb4e31c-afac-4bb7-8c44-41ad3c39ae3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:29.502543 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2cbe2f-6fe7-47be-a797-fc5bb48265df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:29.503359 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2cbe2f-6fe7-47be-a797-fc5bb48265df None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:29.503689 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2cbe2f-6fe7-47be-a797-fc5bb48265df None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:29.503909 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2cbe2f-6fe7-47be-a797-fc5bb48265df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:29.516178 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2cbe2f-6fe7-47be-a797-fc5bb48265df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:29.518190 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2cbe2f-6fe7-47be-a797-fc5bb48265df None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:29.540243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-9d2cbe2f-6fe7-47be-a797-fc5bb48265df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:29.540243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:29.540243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:29.540243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:29.540243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:29.540243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:29.540243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:29.540243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:29.540243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:29.540243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:29.540243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:29.540243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:29.540243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:29.540243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:29.540243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:29.541380 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:29.541380 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:29.541380 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:29.541380 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:29.541380 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:29.541380 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:29.541380 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:29.541380 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:29.541380 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:29.541380 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:29.541380 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:29.541380 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:29.541380 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:29.541380 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:29.541380 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:29.541380 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:29.541380 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:29.542342 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:29.542342 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:29.542342 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:29.542342 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:29.542342 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 365/737] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:29.543125 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79641070-7864-49b3-8ea0-939e4ff3a923 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:29.543228 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79641070-7864-49b3-8ea0-939e4ff3a923 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:29.543516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79641070-7864-49b3-8ea0-939e4ff3a923 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:29.545149 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 366/738] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:29.551656 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79641070-7864-49b3-8ea0-939e4ff3a923 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:29.553743 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 373/739] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:29 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:29.554839 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5632e4-cad9-46c9-96fc-fb3fd281e5a4 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:29.555121 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5632e4-cad9-46c9-96fc-fb3fd281e5a4 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:29.555396 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5632e4-cad9-46c9-96fc-fb3fd281e5a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:29.594818 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-9d5632e4-cad9-46c9-96fc-fb3fd281e5a4 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:29.597906 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-9d5632e4-cad9-46c9-96fc-fb3fd281e5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:29.670644 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79641070-7864-49b3-8ea0-939e4ff3a923 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:29.681106 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 374/740] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:29 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 27 21:03:29.681106 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5632e4-cad9-46c9-96fc-fb3fd281e5a4 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:29.682578 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:29.699464 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:29.720163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b3cc2c-c5a1-4396-834e-98f0e44a99dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:29.720835 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b3cc2c-c5a1-4396-834e-98f0e44a99dd None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:29.721165 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b3cc2c-c5a1-4396-834e-98f0e44a99dd None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:29.721447 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b3cc2c-c5a1-4396-834e-98f0e44a99dd None admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:29.722222 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b3cc2c-c5a1-4396-834e-98f0e44a99dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:29.722530 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b3cc2c-c5a1-4396-834e-98f0e44a99dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:29.724409 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b3cc2c-c5a1-4396-834e-98f0e44a99dd None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:29.737982 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 367/741] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:29.738822 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b3cc2c-c5a1-4396-834e-98f0e44a99dd None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:29.799821 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5632e4-cad9-46c9-96fc-fb3fd281e5a4 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:29.857961 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dac453-4cdf-4df0-8c8a-50494935bbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:29.858992 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dac453-4cdf-4df0-8c8a-50494935bbd2 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:29.859595 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dac453-4cdf-4df0-8c8a-50494935bbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:29.860054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dac453-4cdf-4df0-8c8a-50494935bbd2 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:29.861299 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dac453-4cdf-4df0-8c8a-50494935bbd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:29.861956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dac453-4cdf-4df0-8c8a-50494935bbd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:29.864317 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dac453-4cdf-4df0-8c8a-50494935bbd2 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:29.884729 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b3cc2c-c5a1-4396-834e-98f0e44a99dd None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:29.911725 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 375/742] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:29.911725 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dac453-4cdf-4df0-8c8a-50494935bbd2 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:29.937146 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b60bd40-1f93-4b04-8595-911432141fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:29.937858 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60bd40-1f93-4b04-8595-911432141fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:29.938209 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60bd40-1f93-4b04-8595-911432141fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:29.938514 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60bd40-1f93-4b04-8595-911432141fb0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:29.949966 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b60bd40-1f93-4b04-8595-911432141fb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:29.958372 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-9b60bd40-1f93-4b04-8595-911432141fb0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:29.958372 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:29.958372 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:29.958372 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:29.958372 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:29.958372 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:29.958372 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:29.958372 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:29.958372 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:29.958372 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:29.958372 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:29.958372 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:29.958372 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:29.958372 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:29.958372 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:29.958372 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:29.964959 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:29.964959 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:29.964959 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:29.964959 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:29.964959 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:29.964959 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:29.964959 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:29.964959 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:29.964959 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:29.964959 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:29.964959 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:29.964959 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:29.964959 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:29.964959 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:29.964959 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:29.964959 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:29.966068 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:29.966068 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:29.966068 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:29.966068 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:29.966068 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:29.966068 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:29.966068 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:29.966068 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:29.966068 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:29.966068 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:29.966068 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:29.966068 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:29.966068 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 368/743] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 27 21:03:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:29.966068 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b60bd40-1f93-4b04-8595-911432141fb0 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:29.995376 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:30.005252 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:30.046473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dac453-4cdf-4df0-8c8a-50494935bbd2 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:30.086923 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e5ea3-e84f-4666-94db-cdd4e03516b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:30.087580 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e5ea3-e84f-4666-94db-cdd4e03516b2 None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:30.087863 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e5ea3-e84f-4666-94db-cdd4e03516b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:30.088138 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e5ea3-e84f-4666-94db-cdd4e03516b2 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:30.089525 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e5ea3-e84f-4666-94db-cdd4e03516b2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:30.090919 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e5ea3-e84f-4666-94db-cdd4e03516b2 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:30.097490 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b60bd40-1f93-4b04-8595-911432141fb0 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:30.109101 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 376/744] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Jun 27 21:03:29 2022] POST /identity/v3/services => generated 240 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:03:30.110595 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e5ea3-e84f-4666-94db-cdd4e03516b2 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:30.164533 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a43abb-495b-48bb-90b2-d852d33a6850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:30.165745 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a43abb-495b-48bb-90b2-d852d33a6850 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:30.166182 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a43abb-495b-48bb-90b2-d852d33a6850 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:30.166623 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a43abb-495b-48bb-90b2-d852d33a6850 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:30.175262 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a43abb-495b-48bb-90b2-d852d33a6850 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:30.177187 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a43abb-495b-48bb-90b2-d852d33a6850 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:30.194967 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-b1a43abb-495b-48bb-90b2-d852d33a6850 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:30.194967 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:30.194967 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:30.194967 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:30.194967 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:30.194967 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:30.194967 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:30.194967 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:30.194967 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:30.194967 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:30.194967 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:30.194967 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:30.194967 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:30.194967 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:30.194967 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:30.196127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:30.196127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:30.196127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:30.196127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:30.196127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:30.196127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:30.196127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:30.196127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:30.196127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:30.196127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:30.196127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:30.196127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:30.196127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:30.196127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:30.196127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:30.196127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:30.196127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:30.197227 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:30.197227 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:30.197227 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:30.197227 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:30.197227 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:30.197227 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:30.197227 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:30.197227 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:30.199804 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 369/745] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:30.200911 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a43abb-495b-48bb-90b2-d852d33a6850 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:30.263595 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e5ea3-e84f-4666-94db-cdd4e03516b2 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:30.291424 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:30.304963 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b9340-349b-4280-9d34-576c8c8e5abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:30.305914 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b9340-349b-4280-9d34-576c8c8e5abb None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:30.306490 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b9340-349b-4280-9d34-576c8c8e5abb None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:30.307058 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b9340-349b-4280-9d34-576c8c8e5abb None admin] PATH_INFO: `/v3/users/swift` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:30.317764 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516b9340-349b-4280-9d34-576c8c8e5abb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:30.318973 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516b9340-349b-4280-9d34-576c8c8e5abb None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:30.325401 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:30.333235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-516b9340-349b-4280-9d34-576c8c8e5abb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 27 21:03:30.333235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:30.333235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:30.333235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:30.333235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:30.333235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:30.333235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:30.333235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:30.333235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:30.333235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:30.333235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:30.333235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:30.333235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:30.333235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:30.333235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:30.333235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:30.334204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:30.334204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:30.334204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:30.334204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:30.334204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:30.334204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:30.334204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:30.334204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:30.334204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:30.334204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:30.334204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:30.334204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:30.334204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:30.334204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:30.334204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:30.334204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:30.334204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:30.334204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:30.335234 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:30.335234 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:30.335234 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:30.335234 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:30.335234 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:30.335234 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:30.335234 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:30.335234 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:30.335234 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 27 21:03:30.335234 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:30.335234 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 377/746] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 27 21:03:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:30.335822 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b9340-349b-4280-9d34-576c8c8e5abb None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:30.342757 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a43abb-495b-48bb-90b2-d852d33a6850 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:30.391865 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c9d96-4dfd-40fe-9ba1-b945a9fee758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:30.392564 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c9d96-4dfd-40fe-9ba1-b945a9fee758 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:30.392849 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c9d96-4dfd-40fe-9ba1-b945a9fee758 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:30.393275 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c9d96-4dfd-40fe-9ba1-b945a9fee758 None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:30.394078 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9c9d96-4dfd-40fe-9ba1-b945a9fee758 None admin] RBAC: Adding query filter params (name=service) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:30.394391 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9c9d96-4dfd-40fe-9ba1-b945a9fee758 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:30.395860 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9c9d96-4dfd-40fe-9ba1-b945a9fee758 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:30.408977 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 370/747] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 27 21:03:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:30.408977 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c9d96-4dfd-40fe-9ba1-b945a9fee758 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:30.472487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b9340-349b-4280-9d34-576c8c8e5abb None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:30.520456 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8257df27-b427-4be0-8cd2-7c2ade950d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:30.521714 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257df27-b427-4be0-8cd2-7c2ade950d1f None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:30.522149 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257df27-b427-4be0-8cd2-7c2ade950d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:30.522594 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8257df27-b427-4be0-8cd2-7c2ade950d1f None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:30.523543 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8257df27-b427-4be0-8cd2-7c2ade950d1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:30.524016 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8257df27-b427-4be0-8cd2-7c2ade950d1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:30.524293 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c9d96-4dfd-40fe-9ba1-b945a9fee758 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:30.525872 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8257df27-b427-4be0-8cd2-7c2ade950d1f None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:30.545184 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 378/748] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:30.546989 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8257df27-b427-4be0-8cd2-7c2ade950d1f None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:30.569010 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d850d580-b077-4a87-a894-ae0ff6f60c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:30.569872 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850d580-b077-4a87-a894-ae0ff6f60c4e None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:30.570245 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850d580-b077-4a87-a894-ae0ff6f60c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:30.570609 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850d580-b077-4a87-a894-ae0ff6f60c4e None admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:30.571558 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d850d580-b077-4a87-a894-ae0ff6f60c4e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:30.571970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d850d580-b077-4a87-a894-ae0ff6f60c4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:30.573820 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d850d580-b077-4a87-a894-ae0ff6f60c4e None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:30.597682 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 371/749] 10.222.0.13 () {56 vars in 1135 bytes} [Mon Jun 27 21:03:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:30.598722 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d850d580-b077-4a87-a894-ae0ff6f60c4e None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:30.613930 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:30.618816 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:30.673069 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8257df27-b427-4be0-8cd2-7c2ade950d1f None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:30.712375 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02299979-5017-469a-ae1f-f5417a138632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:30.712766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d850d580-b077-4a87-a894-ae0ff6f60c4e None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:30.713274 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02299979-5017-469a-ae1f-f5417a138632 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:30.713510 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02299979-5017-469a-ae1f-f5417a138632 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:30.713858 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02299979-5017-469a-ae1f-f5417a138632 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:30.726904 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02299979-5017-469a-ae1f-f5417a138632 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:30.728636 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02299979-5017-469a-ae1f-f5417a138632 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:30.747034 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-02299979-5017-469a-ae1f-f5417a138632 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:30.747034 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:30.747034 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:30.747034 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:30.747034 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:30.747034 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:30.747034 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:30.747034 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:30.747034 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:30.747034 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:30.747034 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:30.747034 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:30.747034 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:30.747034 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:30.747034 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:30.748147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:30.748147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:30.748147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:30.748147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:30.748147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:30.748147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:30.748147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:30.748147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:30.748147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:30.748147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:30.748147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:30.748147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:30.748147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:30.748147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:30.748147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:30.748147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:30.748147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:30.749079 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:30.749079 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:30.749079 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:30.749079 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:30.751289 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 379/750] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:30.752490 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02299979-5017-469a-ae1f-f5417a138632 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:30.754433 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6415553a-2c86-49db-beb8-3d50e7992b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:30.755540 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415553a-2c86-49db-beb8-3d50e7992b13 None admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:30.755970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415553a-2c86-49db-beb8-3d50e7992b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:30.756319 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415553a-2c86-49db-beb8-3d50e7992b13 None admin] PATH_INFO: `/v3/projects/334438c929c44e489e9a3b93e6d37231/users/1366958034d540b4ba2feb421a6afabc/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:30.759390 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6415553a-2c86-49db-beb8-3d50e7992b13 None admin] RBAC: Authorizing `identity:create_grant(project_id=334438c929c44e489e9a3b93e6d37231, user_id=1366958034d540b4ba2feb421a6afabc, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:30.761020 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6415553a-2c86-49db-beb8-3d50e7992b13 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:30.779560 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 372/751] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 27 21:03:30 2022] PUT /identity/v3/projects/334438c929c44e489e9a3b93e6d37231/users/1366958034d540b4ba2feb421a6afabc/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:03:30.780701 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6415553a-2c86-49db-beb8-3d50e7992b13 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:30.883068 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:30.891292 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:30.895257 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02299979-5017-469a-ae1f-f5417a138632 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:30.932041 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6415553a-2c86-49db-beb8-3d50e7992b13 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:30.978074 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c60f9-62a1-47b8-ba37-f656a0e546db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:30.979022 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c60f9-62a1-47b8-ba37-f656a0e546db None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:30.979743 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c60f9-62a1-47b8-ba37-f656a0e546db None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:30.980207 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c60f9-62a1-47b8-ba37-f656a0e546db None admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:30.981206 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4c60f9-62a1-47b8-ba37-f656a0e546db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:30.981754 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4c60f9-62a1-47b8-ba37-f656a0e546db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:30.983357 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4c60f9-62a1-47b8-ba37-f656a0e546db None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:30.984183 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2c9f7-2752-47f3-a4ef-1519db293e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:30.984974 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2c9f7-2752-47f3-a4ef-1519db293e36 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:30.985360 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2c9f7-2752-47f3-a4ef-1519db293e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:30.985787 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2c9f7-2752-47f3-a4ef-1519db293e36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:31.005159 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 373/752] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:31.013446 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c60f9-62a1-47b8-ba37-f656a0e546db None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:31.078424 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a2c9f7-2752-47f3-a4ef-1519db293e36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:31.079909 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a2c9f7-2752-47f3-a4ef-1519db293e36 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:31.133923 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c60f9-62a1-47b8-ba37-f656a0e546db None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:31.171186 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:31.178446 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6cf21-ccdc-4cec-82b4-e290d927fb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:31.179583 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6cf21-ccdc-4cec-82b4-e290d927fb18 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:31.180045 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6cf21-ccdc-4cec-82b4-e290d927fb18 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:31.180460 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f6cf21-ccdc-4cec-82b4-e290d927fb18 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:31.182102 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:31.190776 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 380/753] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:30 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 21:03:31.192287 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f6cf21-ccdc-4cec-82b4-e290d927fb18 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:31.194242 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f6cf21-ccdc-4cec-82b4-e290d927fb18 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:31.202225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2c9f7-2752-47f3-a4ef-1519db293e36 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:31.211596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-00f6cf21-ccdc-4cec-82b4-e290d927fb18 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 21:03:31.211596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:31.211596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:31.211596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:31.211596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:31.211596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:31.211596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:31.211596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:31.211596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:31.211596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:31.211596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:31.211596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:31.211596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:31.211596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:31.211596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:31.211596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:31.212538 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:31.212538 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:31.212538 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:31.212538 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:31.212538 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:31.212538 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:31.212538 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:31.212538 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:31.212538 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:31.212538 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:31.212538 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:31.212538 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:31.212538 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:31.212538 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:31.212538 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:31.212538 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:31.212538 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:31.212538 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:31.213593 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:31.213593 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:31.213593 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:31.213593 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:31.213593 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:31.213593 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:31.213593 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:31.213593 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:31.213593 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 21:03:31.213593 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:31.216309 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 374/754] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Jun 27 21:03:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:31.221584 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6cf21-ccdc-4cec-82b4-e290d927fb18 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:31.336995 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f6cf21-ccdc-4cec-82b4-e290d927fb18 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:31.377853 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5213bc3c-fd32-48c1-92ae-905c5db7addb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:31.379173 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213bc3c-fd32-48c1-92ae-905c5db7addb None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:31.379655 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213bc3c-fd32-48c1-92ae-905c5db7addb None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:31.380191 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213bc3c-fd32-48c1-92ae-905c5db7addb None admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:31.381339 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5213bc3c-fd32-48c1-92ae-905c5db7addb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:31.381879 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5213bc3c-fd32-48c1-92ae-905c5db7addb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:31.383628 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5213bc3c-fd32-48c1-92ae-905c5db7addb None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:31.408344 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 375/755] 10.222.0.13 () {56 vars in 1149 bytes} [Mon Jun 27 21:03:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:31.413851 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5213bc3c-fd32-48c1-92ae-905c5db7addb None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:31.453141 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2c9f7-2752-47f3-a4ef-1519db293e36 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:31.518582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:31.521467 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e38ba-4518-43c5-85d5-814a5aa4dd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:31.522518 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e38ba-4518-43c5-85d5-814a5aa4dd7a None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:31.523012 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e38ba-4518-43c5-85d5-814a5aa4dd7a None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:31.524211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e38ba-4518-43c5-85d5-814a5aa4dd7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:31.572940 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30e38ba-4518-43c5-85d5-814a5aa4dd7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:31.576037 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30e38ba-4518-43c5-85d5-814a5aa4dd7a None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:31.616169 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-f30e38ba-4518-43c5-85d5-814a5aa4dd7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:31.616169 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:31.616169 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:31.616169 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:31.616169 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:31.616169 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:31.616169 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:31.616169 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:31.616169 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:31.616169 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:31.616169 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:31.616169 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:31.616169 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:31.616169 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:31.616169 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:31.617938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:31.617938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:31.617938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:31.617938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:31.617938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:31.617938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:31.617938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:31.617938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:31.617938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:31.617938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:31.617938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:31.617938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:31.617938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:31.617938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:31.617938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:31.617938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:31.617938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:31.619185 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:31.619185 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:31.619185 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:31.619185 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:31.619185 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 381/756] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 416 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:31.619185 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe1ead-b9b1-45e4-949b-c22032ef8390 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:31.619185 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe1ead-b9b1-45e4-949b-c22032ef8390 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:31.619608 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe1ead-b9b1-45e4-949b-c22032ef8390 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:31.621396 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 382/757] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:31.626122 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe1ead-b9b1-45e4-949b-c22032ef8390 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:31.650227 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5213bc3c-fd32-48c1-92ae-905c5db7addb None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:31.687295 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:31.701497 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b57dbc-0b03-415d-bac1-ac5d7cc35209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:31.702268 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b57dbc-0b03-415d-bac1-ac5d7cc35209 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:31.702473 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b57dbc-0b03-415d-bac1-ac5d7cc35209 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:31.702727 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b57dbc-0b03-415d-bac1-ac5d7cc35209 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:31.754037 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe1ead-b9b1-45e4-949b-c22032ef8390 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:31.796449 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9589b0-0681-4c82-b7df-456b1b679a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:31.797150 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9589b0-0681-4c82-b7df-456b1b679a0a None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:31.797430 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9589b0-0681-4c82-b7df-456b1b679a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:31.797735 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9589b0-0681-4c82-b7df-456b1b679a0a None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:31.798584 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9589b0-0681-4c82-b7df-456b1b679a0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:31.798894 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9589b0-0681-4c82-b7df-456b1b679a0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:31.800944 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9589b0-0681-4c82-b7df-456b1b679a0a None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:31.802057 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b57dbc-0b03-415d-bac1-ac5d7cc35209 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:31.804819 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b57dbc-0b03-415d-bac1-ac5d7cc35209 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:31.821350 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 383/758] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:31.821350 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d1def-9153-48e7-8eec-59d0c6dd9529 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:31.821350 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d1def-9153-48e7-8eec-59d0c6dd9529 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:31.821350 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d1def-9153-48e7-8eec-59d0c6dd9529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:31.844314 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-259d1def-9153-48e7-8eec-59d0c6dd9529 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:31.845292 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-259d1def-9153-48e7-8eec-59d0c6dd9529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:31.878414 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:31.952949 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 376/760] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:31 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 21:03:31.952949 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b57dbc-0b03-415d-bac1-ac5d7cc35209 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:31.953486 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 384/760] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:03:31.954696 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259d1def-9153-48e7-8eec-59d0c6dd9529 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:32.016015 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:32.064654 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b57dbc-0b03-415d-bac1-ac5d7cc35209 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:32.081442 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259d1def-9153-48e7-8eec-59d0c6dd9529 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:32.107946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137bf6d-524b-48c3-8263-47a44efecb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:32.108453 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137bf6d-524b-48c3-8263-47a44efecb68 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:32.108735 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137bf6d-524b-48c3-8263-47a44efecb68 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:32.109120 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137bf6d-524b-48c3-8263-47a44efecb68 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:32.117617 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c137bf6d-524b-48c3-8263-47a44efecb68 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:32.119093 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c137bf6d-524b-48c3-8263-47a44efecb68 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:32.126335 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a243c2-8ec9-47b8-9fcc-151d72126dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:32.127553 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a243c2-8ec9-47b8-9fcc-151d72126dcc None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:32.128481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a243c2-8ec9-47b8-9fcc-151d72126dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:32.129044 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a243c2-8ec9-47b8-9fcc-151d72126dcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:32.137767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-c137bf6d-524b-48c3-8263-47a44efecb68 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:32.137767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:32.137767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:32.137767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:32.137767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:32.137767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:32.137767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:32.137767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:32.137767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:32.137767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:32.137767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:32.137767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:32.137767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:32.137767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:32.137767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:32.138756 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:32.138756 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:32.138756 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:32.138756 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:32.138756 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:32.138756 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:32.138756 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:32.138756 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:32.138756 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:32.138756 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:32.138756 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:32.138756 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:32.138756 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:32.138756 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:32.138756 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:32.138756 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:32.138756 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:32.140121 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:32.140121 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:32.140121 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:32.140121 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:32.140121 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:32.140121 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:32.140121 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:32.140121 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:32.143749 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 377/761] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:32.144798 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137bf6d-524b-48c3-8263-47a44efecb68 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:32.146053 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a243c2-8ec9-47b8-9fcc-151d72126dcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:32.148059 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a243c2-8ec9-47b8-9fcc-151d72126dcc None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:32.159115 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:32.174531 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-15a243c2-8ec9-47b8-9fcc-151d72126dcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:32.174531 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:32.174531 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:32.174531 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:32.174531 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:32.174531 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:32.174531 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:32.174531 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:32.174531 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:32.174531 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:32.174531 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:32.174531 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:32.174531 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:32.174531 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:32.174531 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:32.175680 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:32.175680 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:32.175680 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:32.175680 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:32.175680 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:32.175680 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:32.175680 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:32.175680 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:32.175680 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:32.175680 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:32.175680 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:32.175680 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:32.175680 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:32.175680 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:32.175680 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:32.175680 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:32.175680 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:32.176775 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:32.176775 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:32.176775 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:32.176775 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:32.179396 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 385/762] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:32.180440 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a243c2-8ec9-47b8-9fcc-151d72126dcc None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:32.271407 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137bf6d-524b-48c3-8263-47a44efecb68 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:32.296874 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:32.307684 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a243c2-8ec9-47b8-9fcc-151d72126dcc None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:32.309528 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7907b22b-4d21-4691-96b5-438b3881597c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:32.310474 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7907b22b-4d21-4691-96b5-438b3881597c None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:32.310854 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7907b22b-4d21-4691-96b5-438b3881597c None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:32.311884 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7907b22b-4d21-4691-96b5-438b3881597c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:32.311884 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7907b22b-4d21-4691-96b5-438b3881597c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:32.313686 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7907b22b-4d21-4691-96b5-438b3881597c None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:32.328615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-7907b22b-4d21-4691-96b5-438b3881597c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 21:03:32.328615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:32.328615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:03:32.328615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:03:32.328615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:03:32.328615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:03:32.328615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:03:32.328615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:32.328615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 21:03:32.328615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:32.328615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:03:32.328615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:32.328615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:32.328615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:32.328615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:32.328615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:32.329871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:32.329871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:32.329871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:32.329871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:32.329871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:32.329871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:32.329871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:32.329871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:03:32.329871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:03:32.329871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:03:32.329871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:03:32.329871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:32.329871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:32.329871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:32.329871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:32.329871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:32.330903 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:32.330903 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:32.330903 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:32.330903 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:32.330903 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:32.330903 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:32.330903 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:32.330903 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:32.330903 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:32.330903 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:32.330903 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:32.330903 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:03:32.330903 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:32.330903 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 21:03:32.330903 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:32.337224 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 378/763] 10.222.0.13 () {56 vars in 1129 bytes} [Mon Jun 27 21:03:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:32.338301 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7907b22b-4d21-4691-96b5-438b3881597c None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:32.360594 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3cb1a1-ffe9-40d5-90fc-04c5d348e605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:32.361587 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3cb1a1-ffe9-40d5-90fc-04c5d348e605 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:32.362011 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3cb1a1-ffe9-40d5-90fc-04c5d348e605 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:32.362426 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3cb1a1-ffe9-40d5-90fc-04c5d348e605 None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:32.363375 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3cb1a1-ffe9-40d5-90fc-04c5d348e605 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:32.363857 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3cb1a1-ffe9-40d5-90fc-04c5d348e605 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:32.365524 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3cb1a1-ffe9-40d5-90fc-04c5d348e605 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:32.392036 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 386/764] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:32.392849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3cb1a1-ffe9-40d5-90fc-04c5d348e605 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:32.432616 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:32.466352 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7907b22b-4d21-4691-96b5-438b3881597c None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:32.507895 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa1bf43-1ba5-4627-88f0-946146cd2d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:32.508578 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1bf43-1ba5-4627-88f0-946146cd2d9d None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:32.508871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1bf43-1ba5-4627-88f0-946146cd2d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:32.509179 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1bf43-1ba5-4627-88f0-946146cd2d9d None admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:32.509913 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa1bf43-1ba5-4627-88f0-946146cd2d9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:32.510221 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa1bf43-1ba5-4627-88f0-946146cd2d9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:32.511846 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa1bf43-1ba5-4627-88f0-946146cd2d9d None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:32.525346 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 379/765] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:32.526992 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df797b-da3c-4454-99f7-9b346bfbaa3e None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:32.527284 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df797b-da3c-4454-99f7-9b346bfbaa3e None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:32.527559 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df797b-da3c-4454-99f7-9b346bfbaa3e None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:32.528642 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3cb1a1-ffe9-40d5-90fc-04c5d348e605 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:32.529160 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 380/766] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:32.529884 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df797b-da3c-4454-99f7-9b346bfbaa3e None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:32.574416 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77619bf5-5add-4685-8bae-f19ffc76855c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:32.575094 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77619bf5-5add-4685-8bae-f19ffc76855c None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:32.575373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77619bf5-5add-4685-8bae-f19ffc76855c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:32.575645 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77619bf5-5add-4685-8bae-f19ffc76855c None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:32.576364 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77619bf5-5add-4685-8bae-f19ffc76855c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:32.576668 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77619bf5-5add-4685-8bae-f19ffc76855c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:32.578127 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77619bf5-5add-4685-8bae-f19ffc76855c None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:32.583198 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 387/767] 10.222.0.13 () {56 vars in 1124 bytes} [Mon Jun 27 21:03:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:32.584014 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77619bf5-5add-4685-8bae-f19ffc76855c None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:32.603162 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:32.667476 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df797b-da3c-4454-99f7-9b346bfbaa3e None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:32.716552 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77619bf5-5add-4685-8bae-f19ffc76855c None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:32.717157 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979206a0-fa90-4e29-b000-085ed382f1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:32.718026 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979206a0-fa90-4e29-b000-085ed382f1b9 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:32.718438 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979206a0-fa90-4e29-b000-085ed382f1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:32.718777 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979206a0-fa90-4e29-b000-085ed382f1b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:32.720012 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979206a0-fa90-4e29-b000-085ed382f1b9 None admin] RBAC: Adding query filter params (role.id=d336dd7b746c4488b33289c00302e4d5, scope.project.id=334438c929c44e489e9a3b93e6d37231, user.id=8bed6b231ee647d8a350da055d8e59d7) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:32.720504 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979206a0-fa90-4e29-b000-085ed382f1b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:32.722462 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979206a0-fa90-4e29-b000-085ed382f1b9 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:32.725420 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:32.741302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 381/768] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:03:32 2022] GET /identity/v3/role_assignments?user.id=8bed6b231ee647d8a350da055d8e59d7&scope.project.id=334438c929c44e489e9a3b93e6d37231&role.id=d336dd7b746c4488b33289c00302e4d5 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:32.742877 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd9bde-649c-48d5-827c-87713e072205 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:32.743223 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd9bde-649c-48d5-827c-87713e072205 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:32.743502 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd9bde-649c-48d5-827c-87713e072205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:32.775364 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbeefeb-3861-416b-bfd9-d61e8b1aeffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:32.776507 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbeefeb-3861-416b-bfd9-d61e8b1aeffb None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:32.777082 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbeefeb-3861-416b-bfd9-d61e8b1aeffb None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:32.777177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbeefeb-3861-416b-bfd9-d61e8b1aeffb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:32.782760 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-fddd9bde-649c-48d5-827c-87713e072205 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:32.784087 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-fddd9bde-649c-48d5-827c-87713e072205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:32.787719 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbeefeb-3861-416b-bfd9-d61e8b1aeffb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:32.789726 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbeefeb-3861-416b-bfd9-d61e8b1aeffb None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:32.810567 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-4fbeefeb-3861-416b-bfd9-d61e8b1aeffb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:32.810567 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:32.810567 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:32.810567 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:32.810567 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:32.810567 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:32.810567 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:32.810567 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:32.810567 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:32.810567 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:32.810567 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:32.810567 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:32.810567 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:32.810567 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:32.810567 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:32.811556 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:32.811556 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:32.811556 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:32.811556 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:32.811556 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:32.811556 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:32.811556 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:32.811556 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:32.811556 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:32.811556 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:32.811556 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:32.811556 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:32.811556 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:32.811556 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:32.811556 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:32.811556 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:32.811556 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:32.812483 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:32.812483 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:32.812483 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:32.812483 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:32.812483 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:32.812483 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:32.812483 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:32.812483 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:32.818283 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 388/769] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:32.819442 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbeefeb-3861-416b-bfd9-d61e8b1aeffb None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:32.893430 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 382/770] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:03:32.894625 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd9bde-649c-48d5-827c-87713e072205 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:32.941899 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:32.974008 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbeefeb-3861-416b-bfd9-d61e8b1aeffb None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:33.028634 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b09e5b-1c47-4f77-bd9e-d17c1b3a54db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:33.029635 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b09e5b-1c47-4f77-bd9e-d17c1b3a54db None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:33.029917 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b09e5b-1c47-4f77-bd9e-d17c1b3a54db None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:33.030193 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b09e5b-1c47-4f77-bd9e-d17c1b3a54db None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:33.030974 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b09e5b-1c47-4f77-bd9e-d17c1b3a54db None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:33.031285 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b09e5b-1c47-4f77-bd9e-d17c1b3a54db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:33.032768 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b09e5b-1c47-4f77-bd9e-d17c1b3a54db None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:33.047316 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 389/771] 10.222.0.13 () {56 vars in 1124 bytes} [Mon Jun 27 21:03:32 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:33.048308 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b09e5b-1c47-4f77-bd9e-d17c1b3a54db None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:33.059330 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd9bde-649c-48d5-827c-87713e072205 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:33.091506 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:33.102137 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e817a90-dbc7-4587-b376-a1bb13a32666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:33.103200 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e817a90-dbc7-4587-b376-a1bb13a32666 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:33.103540 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e817a90-dbc7-4587-b376-a1bb13a32666 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:33.103920 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e817a90-dbc7-4587-b376-a1bb13a32666 None admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:33.105500 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e817a90-dbc7-4587-b376-a1bb13a32666 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:33.105933 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e817a90-dbc7-4587-b376-a1bb13a32666 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:33.107486 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e817a90-dbc7-4587-b376-a1bb13a32666 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:33.121016 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 383/772] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:33.122019 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e817a90-dbc7-4587-b376-a1bb13a32666 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:33.186733 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b09e5b-1c47-4f77-bd9e-d17c1b3a54db None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:33.225736 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd400a7-1b0b-4546-9f7a-e82b2f515a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:33.226817 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd400a7-1b0b-4546-9f7a-e82b2f515a01 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:33.227096 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd400a7-1b0b-4546-9f7a-e82b2f515a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:33.227367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd400a7-1b0b-4546-9f7a-e82b2f515a01 None admin] PATH_INFO: `/v3/services/network` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:33.228047 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd400a7-1b0b-4546-9f7a-e82b2f515a01 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:33.229538 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd400a7-1b0b-4546-9f7a-e82b2f515a01 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:33.238300 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e817a90-dbc7-4587-b376-a1bb13a32666 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:33.243498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-8fd400a7-1b0b-4546-9f7a-e82b2f515a01 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 21:03:33.243498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:33.243498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:03:33.243498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:03:33.243498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:03:33.243498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:03:33.243498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:03:33.243498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:33.243498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 21:03:33.243498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:33.243498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:03:33.243498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:33.243498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:33.243498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:33.243498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:33.243498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:33.243498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:33.244481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:33.244481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:33.244481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:33.244481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:33.244481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:33.244481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:33.244481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:03:33.244481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:03:33.244481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:03:33.244481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:03:33.244481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:33.244481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:33.244481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:33.244481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:33.244481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:33.244481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:33.245394 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:33.245394 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:33.245394 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:33.245394 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:33.245394 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:33.245394 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:33.245394 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:33.245394 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:33.245394 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:33.245394 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:33.245394 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:03:33.245394 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:33.245394 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 21:03:33.245394 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:33.250854 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 390/773] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:33 2022] GET /identity/v3/services/network => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:33.250854 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd400a7-1b0b-4546-9f7a-e82b2f515a01 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:33.253681 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:33.279744 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec08f9-ff1f-42de-8bb6-f38a992ebebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:33.280428 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec08f9-ff1f-42de-8bb6-f38a992ebebd None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:33.280710 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec08f9-ff1f-42de-8bb6-f38a992ebebd None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:33.281023 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec08f9-ff1f-42de-8bb6-f38a992ebebd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:33.281837 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec08f9-ff1f-42de-8bb6-f38a992ebebd None admin] RBAC: Adding query filter params (interface=public, service_id=13cea514e2df48b7847a417807ed9e36, region_id=RegionOne) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:33.282150 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec08f9-ff1f-42de-8bb6-f38a992ebebd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:33.283542 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec08f9-ff1f-42de-8bb6-f38a992ebebd None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:33.294102 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 384/774] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Jun 27 21:03:33 2022] GET /identity/v3/endpoints?service_id=13cea514e2df48b7847a417807ed9e36&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:33.294102 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0a731-1d01-4395-b41d-22c84b563700 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:33.294102 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0a731-1d01-4395-b41d-22c84b563700 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:33.294102 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0a731-1d01-4395-b41d-22c84b563700 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:33.295340 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 385/775] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:33.297751 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a0a731-1d01-4395-b41d-22c84b563700 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:33.366759 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:33.367452 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd400a7-1b0b-4546-9f7a-e82b2f515a01 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:33.404445 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57cac27-9a33-4160-9935-3bc30d7def08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:33.405140 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57cac27-9a33-4160-9935-3bc30d7def08 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:33.405421 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57cac27-9a33-4160-9935-3bc30d7def08 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:33.405695 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57cac27-9a33-4160-9935-3bc30d7def08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:33.406538 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57cac27-9a33-4160-9935-3bc30d7def08 None admin] RBAC: Adding query filter params (role.id=709aef6b85524eb2a8e9986a2bd9525b, scope.project.id=334438c929c44e489e9a3b93e6d37231, user.id=f67110d591a94537a3df6a2d835370f9) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:33.406850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57cac27-9a33-4160-9935-3bc30d7def08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:33.408316 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57cac27-9a33-4160-9935-3bc30d7def08 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:33.414309 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 391/776] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:03:33 2022] GET /identity/v3/role_assignments?user.id=f67110d591a94537a3df6a2d835370f9&scope.project.id=334438c929c44e489e9a3b93e6d37231&role.id=709aef6b85524eb2a8e9986a2bd9525b => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:33.415126 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57cac27-9a33-4160-9935-3bc30d7def08 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:33.433557 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a0a731-1d01-4395-b41d-22c84b563700 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:33.480166 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465d69c4-98f8-4226-bad3-1e5fa084d262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:33.480840 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d69c4-98f8-4226-bad3-1e5fa084d262 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:33.481129 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d69c4-98f8-4226-bad3-1e5fa084d262 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:33.481450 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d69c4-98f8-4226-bad3-1e5fa084d262 None admin] PATH_INFO: `/v3/services` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:33.482187 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465d69c4-98f8-4226-bad3-1e5fa084d262 None admin] RBAC: Adding query filter params (name=network) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:33.482550 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465d69c4-98f8-4226-bad3-1e5fa084d262 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:33.484131 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465d69c4-98f8-4226-bad3-1e5fa084d262 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:33.490326 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 386/777] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Jun 27 21:03:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:33.491722 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1c716-63a3-4b02-a636-406aa5b0ce0d None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:33.492009 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1c716-63a3-4b02-a636-406aa5b0ce0d None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:33.492273 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf1c716-63a3-4b02-a636-406aa5b0ce0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:33.511616 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-2cf1c716-63a3-4b02-a636-406aa5b0ce0d None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:33.512905 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:33.513702 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-2cf1c716-63a3-4b02-a636-406aa5b0ce0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:33.563094 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57cac27-9a33-4160-9935-3bc30d7def08 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:33.601806 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 387/778] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:03:33.604167 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5981f47-2c83-443d-9d52-7a0ad341b159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:33.605166 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5981f47-2c83-443d-9d52-7a0ad341b159 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:33.605521 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5981f47-2c83-443d-9d52-7a0ad341b159 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:33.605892 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5981f47-2c83-443d-9d52-7a0ad341b159 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:33.606097 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1c716-63a3-4b02-a636-406aa5b0ce0d None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:33.606717 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5981f47-2c83-443d-9d52-7a0ad341b159 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:33.608217 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5981f47-2c83-443d-9d52-7a0ad341b159 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:33.615660 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 392/779] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Jun 27 21:03:33 2022] GET /identity/v3/services => generated 772 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:33.633279 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5981f47-2c83-443d-9d52-7a0ad341b159 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:33.664309 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:33.729036 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf1c716-63a3-4b02-a636-406aa5b0ce0d None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:33.760072 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5981f47-2c83-443d-9d52-7a0ad341b159 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:33.795216 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:33.807411 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a38d9-14aa-4544-9c39-90f05c45eef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:33.808206 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a38d9-14aa-4544-9c39-90f05c45eef1 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:33.808585 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a38d9-14aa-4544-9c39-90f05c45eef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:33.808860 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a38d9-14aa-4544-9c39-90f05c45eef1 None admin] PATH_INFO: `/v3/services` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:33.809637 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a38d9-14aa-4544-9c39-90f05c45eef1 None admin] RBAC: Adding query filter params (type=network) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:33.809943 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a38d9-14aa-4544-9c39-90f05c45eef1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:33.811379 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a38d9-14aa-4544-9c39-90f05c45eef1 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:33.812233 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b58e647-a506-4e7e-a402-58f55bdb000d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:33.813220 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58e647-a506-4e7e-a402-58f55bdb000d None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:33.813504 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58e647-a506-4e7e-a402-58f55bdb000d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:33.813780 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58e647-a506-4e7e-a402-58f55bdb000d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:33.824094 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 388/780] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Jun 27 21:03:33 2022] GET /identity/v3/services?type=network => generated 350 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:33.830409 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a38d9-14aa-4544-9c39-90f05c45eef1 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:33.837182 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b58e647-a506-4e7e-a402-58f55bdb000d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:33.854215 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-0b58e647-a506-4e7e-a402-58f55bdb000d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:03:33.854215 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:33.854215 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:33.854215 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:33.854215 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:33.854215 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:33.854215 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:33.854215 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:33.854215 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:33.854215 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:33.854215 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:33.854215 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:33.854215 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:33.854215 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:33.854215 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:33.854215 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:33.855259 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:33.855259 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:33.855259 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:33.855259 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:33.855259 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:33.855259 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:33.855259 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:33.855259 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:33.855259 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:33.855259 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:33.855259 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:33.855259 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:33.855259 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:33.855259 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:33.855259 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:33.855259 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:33.856128 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:33.856128 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:33.856128 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:33.856128 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:33.856128 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:33.856128 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:33.856128 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:33.856128 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:33.856128 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:33.856128 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:33.856128 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:03:33.856128 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:33.856128 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 393/781] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 27 21:03:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:33.861822 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b58e647-a506-4e7e-a402-58f55bdb000d None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:33.951779 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a38d9-14aa-4544-9c39-90f05c45eef1 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:33.965907 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:33.991111 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b58e647-a506-4e7e-a402-58f55bdb000d None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:33.993878 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792be797-baae-4071-b72a-b1eb788c44f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:33.994863 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792be797-baae-4071-b72a-b1eb788c44f8 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:33.995358 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792be797-baae-4071-b72a-b1eb788c44f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:33.995936 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792be797-baae-4071-b72a-b1eb788c44f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:33.996970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792be797-baae-4071-b72a-b1eb788c44f8 None admin] RBAC: Adding query filter params (interface=public, service_id=50c68c050f9647678e352ec6803e034a, region_id=RegionOne) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:33.997500 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792be797-baae-4071-b72a-b1eb788c44f8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:33.999205 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792be797-baae-4071-b72a-b1eb788c44f8 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:34.005860 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 389/782] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Jun 27 21:03:33 2022] GET /identity/v3/endpoints?service_id=50c68c050f9647678e352ec6803e034a&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:34.018488 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792be797-baae-4071-b72a-b1eb788c44f8 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:34.032432 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6ac6dc-a472-4338-bc9e-e86552763a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:34.033156 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ac6dc-a472-4338-bc9e-e86552763a6c None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:34.033440 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ac6dc-a472-4338-bc9e-e86552763a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:34.033714 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ac6dc-a472-4338-bc9e-e86552763a6c None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:34.034485 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6ac6dc-a472-4338-bc9e-e86552763a6c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:34.034791 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6ac6dc-a472-4338-bc9e-e86552763a6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:34.036192 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6ac6dc-a472-4338-bc9e-e86552763a6c None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:34.046742 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 394/783] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:03:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:34.053189 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6ac6dc-a472-4338-bc9e-e86552763a6c None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:34.110867 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:34.137757 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792be797-baae-4071-b72a-b1eb788c44f8 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:34.160663 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6ac6dc-a472-4338-bc9e-e86552763a6c None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:34.175631 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f002b2c5-6522-41fa-adae-9e7228dfe6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:34.176674 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002b2c5-6522-41fa-adae-9e7228dfe6fe None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:34.177186 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002b2c5-6522-41fa-adae-9e7228dfe6fe None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:34.177683 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002b2c5-6522-41fa-adae-9e7228dfe6fe None admin] PATH_INFO: `/v3/services` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:34.178648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f002b2c5-6522-41fa-adae-9e7228dfe6fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:34.180302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f002b2c5-6522-41fa-adae-9e7228dfe6fe None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:34.186705 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 390/784] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Jun 27 21:03:34 2022] GET /identity/v3/services => generated 772 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:34.196787 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12751d-b9f7-4afc-913f-d7958dbcd9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:34.197651 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12751d-b9f7-4afc-913f-d7958dbcd9e7 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:34.198004 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12751d-b9f7-4afc-913f-d7958dbcd9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:34.198281 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12751d-b9f7-4afc-913f-d7958dbcd9e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:34.211528 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12751d-b9f7-4afc-913f-d7958dbcd9e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:34.213076 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12751d-b9f7-4afc-913f-d7958dbcd9e7 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:34.233673 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-ef12751d-b9f7-4afc-913f-d7958dbcd9e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:34.233673 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:34.233673 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:34.233673 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:34.233673 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:34.233673 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:34.233673 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:34.233673 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:34.233673 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:34.233673 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:34.233673 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:34.233673 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:34.233673 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:34.233673 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:34.233673 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:34.234867 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:34.234867 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:34.234867 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:34.234867 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:34.234867 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:34.234867 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:34.234867 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:34.234867 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:34.234867 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:34.234867 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:34.234867 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:34.234867 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:34.234867 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:34.234867 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:34.234867 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:34.234867 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:34.234867 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:34.235975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:34.235975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:34.235975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:34.235975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:34.235975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 395/785] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:34.240994 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f002b2c5-6522-41fa-adae-9e7228dfe6fe None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:34.309422 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:34.412870 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f002b2c5-6522-41fa-adae-9e7228dfe6fe None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:34.451831 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47bbc2c-60a1-4c30-82d9-ef7da03d0874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:34.452522 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47bbc2c-60a1-4c30-82d9-ef7da03d0874 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:34.452807 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47bbc2c-60a1-4c30-82d9-ef7da03d0874 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:34.453135 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47bbc2c-60a1-4c30-82d9-ef7da03d0874 None admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:34.453900 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47bbc2c-60a1-4c30-82d9-ef7da03d0874 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:34.454211 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47bbc2c-60a1-4c30-82d9-ef7da03d0874 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:34.455612 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47bbc2c-60a1-4c30-82d9-ef7da03d0874 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:34.469208 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 391/786] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:34.474804 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12751d-b9f7-4afc-913f-d7958dbcd9e7 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:34.582402 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12751d-b9f7-4afc-913f-d7958dbcd9e7 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:34.584814 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:34.618120 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bfb531-074a-42d7-9e00-4acbdecf8c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:34.618936 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfb531-074a-42d7-9e00-4acbdecf8c28 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:34.619488 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfb531-074a-42d7-9e00-4acbdecf8c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:34.619829 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfb531-074a-42d7-9e00-4acbdecf8c28 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:34.627897 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bfb531-074a-42d7-9e00-4acbdecf8c28 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:34.629417 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bfb531-074a-42d7-9e00-4acbdecf8c28 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:34.644523 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-49bfb531-074a-42d7-9e00-4acbdecf8c28 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 21:03:34.644523 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:34.644523 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:34.644523 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:34.644523 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:34.644523 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:34.644523 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:34.644523 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:34.644523 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:34.644523 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:34.644523 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:34.644523 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:34.644523 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:34.644523 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:34.644523 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:34.644523 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:34.645560 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:34.645560 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:34.645560 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:34.645560 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:34.645560 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:34.645560 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:34.645560 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:34.645560 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:34.645560 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:34.645560 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:34.645560 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:34.645560 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:34.645560 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:34.645560 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:34.645560 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:34.645560 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:34.645560 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:34.645560 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:34.646507 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:34.646507 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:34.646507 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:34.646507 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:34.646507 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:34.646507 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:34.646507 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:34.646507 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:34.646507 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 21:03:34.646507 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:34.646507 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 396/787] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Jun 27 21:03:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:34.651978 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47bbc2c-60a1-4c30-82d9-ef7da03d0874 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:34.779250 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:34.783691 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47bbc2c-60a1-4c30-82d9-ef7da03d0874 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:34.820205 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49285928-d8dc-4993-af94-67e3c261e0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:34.820894 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49285928-d8dc-4993-af94-67e3c261e0af None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:34.821474 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49285928-d8dc-4993-af94-67e3c261e0af None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:34.821770 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49285928-d8dc-4993-af94-67e3c261e0af None admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:34.822634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49285928-d8dc-4993-af94-67e3c261e0af None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:34.823077 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49285928-d8dc-4993-af94-67e3c261e0af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:34.824866 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49285928-d8dc-4993-af94-67e3c261e0af None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:34.844648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 392/788] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Jun 27 21:03:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:34.850210 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bfb531-074a-42d7-9e00-4acbdecf8c28 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:34.956192 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bfb531-074a-42d7-9e00-4acbdecf8c28 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:34.992446 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2743a1-ee88-482d-9349-cbab5540ca4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:34.993176 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2743a1-ee88-482d-9349-cbab5540ca4c None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:34.993461 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2743a1-ee88-482d-9349-cbab5540ca4c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:34.993736 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2743a1-ee88-482d-9349-cbab5540ca4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:35.029456 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:35.071996 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2743a1-ee88-482d-9349-cbab5540ca4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:35.073565 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2743a1-ee88-482d-9349-cbab5540ca4c None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:35.135540 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922854a3-31f7-48da-8eff-a3d112504cbb None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:35.136045 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922854a3-31f7-48da-8eff-a3d112504cbb None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:35.136251 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922854a3-31f7-48da-8eff-a3d112504cbb None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:35.137567 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 393/789] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:03:35.143227 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95115cb-a37c-478b-9939-0a6a437a3e45 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:35.143507 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95115cb-a37c-478b-9939-0a6a437a3e45 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:35.143784 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95115cb-a37c-478b-9939-0a6a437a3e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:35.176517 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-c95115cb-a37c-478b-9939-0a6a437a3e45 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:35.177895 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-c95115cb-a37c-478b-9939-0a6a437a3e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:35.223681 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 397/790] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:34 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 21:03:35.230355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2743a1-ee88-482d-9349-cbab5540ca4c None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:35.253445 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 394/791] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:03:35.299059 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95115cb-a37c-478b-9939-0a6a437a3e45 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:35.345365 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:35.381817 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2743a1-ee88-482d-9349-cbab5540ca4c None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:35.429854 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:35.447337 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95115cb-a37c-478b-9939-0a6a437a3e45 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:35.449519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dbdec4-f82d-4d1f-a743-f12aebfc9803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:35.450431 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbdec4-f82d-4d1f-a743-f12aebfc9803 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:35.450715 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbdec4-f82d-4d1f-a743-f12aebfc9803 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:35.450989 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbdec4-f82d-4d1f-a743-f12aebfc9803 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:35.470296 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dbdec4-f82d-4d1f-a743-f12aebfc9803 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:35.471930 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dbdec4-f82d-4d1f-a743-f12aebfc9803 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:35.484053 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1fd308-286b-4024-b222-2c6f37aa8cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:35.484869 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1fd308-286b-4024-b222-2c6f37aa8cee None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:35.485175 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1fd308-286b-4024-b222-2c6f37aa8cee None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:35.485449 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1fd308-286b-4024-b222-2c6f37aa8cee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:35.496104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1fd308-286b-4024-b222-2c6f37aa8cee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:35.498436 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-99dbdec4-f82d-4d1f-a743-f12aebfc9803 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:35.498436 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:35.498436 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:35.498436 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:35.498436 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:35.498436 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:35.498436 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:35.498436 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:35.498436 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:35.498436 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:35.498436 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:35.498436 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:35.498436 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:35.498436 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:35.498436 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:35.499313 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:35.499313 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:35.499313 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:35.499313 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:35.499313 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:35.499313 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:35.499313 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:35.499313 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:35.499313 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:35.499313 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:35.499313 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:35.499313 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:35.499313 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:35.499313 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:35.499313 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:35.499313 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:35.499313 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:35.500198 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:35.500198 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:35.500198 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:35.500198 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:35.500198 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 398/792] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:35.503724 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-7a1fd308-286b-4024-b222-2c6f37aa8cee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:03:35.503724 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:35.503724 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:35.503724 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:35.503724 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:35.503724 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:35.503724 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:35.503724 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:35.503724 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:35.503724 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:35.503724 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:35.503724 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:35.503724 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:35.503724 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:35.503724 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:35.503724 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:35.504987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:35.504987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:35.504987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:35.504987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:35.504987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:35.504987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:35.504987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:35.504987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:35.504987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:35.504987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:35.504987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:35.504987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:35.504987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:35.504987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:35.504987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:35.504987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:35.506911 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:35.506911 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:35.506911 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:35.506911 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:35.506911 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:35.506911 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:35.506911 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:35.506911 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:35.506911 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:35.506911 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:35.506911 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:03:35.506911 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:35.508324 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dbdec4-f82d-4d1f-a743-f12aebfc9803 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:35.508463 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 395/793] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 27 21:03:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:35.511460 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1fd308-286b-4024-b222-2c6f37aa8cee None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:35.615648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1fd308-286b-4024-b222-2c6f37aa8cee None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:35.650429 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9293320d-4e9a-45cf-8154-ab2f7dc24f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:35.651158 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9293320d-4e9a-45cf-8154-ab2f7dc24f8c None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:35.651484 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9293320d-4e9a-45cf-8154-ab2f7dc24f8c None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:35.651822 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9293320d-4e9a-45cf-8154-ab2f7dc24f8c None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:35.652580 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9293320d-4e9a-45cf-8154-ab2f7dc24f8c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:35.652888 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9293320d-4e9a-45cf-8154-ab2f7dc24f8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:35.654372 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9293320d-4e9a-45cf-8154-ab2f7dc24f8c None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:35.663433 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 396/794] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:03:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:35.669006 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9293320d-4e9a-45cf-8154-ab2f7dc24f8c None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:35.697414 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:35.722059 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dbdec4-f82d-4d1f-a743-f12aebfc9803 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:35.772698 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9293320d-4e9a-45cf-8154-ab2f7dc24f8c None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:35.783074 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb45752-91b0-4a3a-a21f-a335bff2b8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:35.783870 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb45752-91b0-4a3a-a21f-a335bff2b8ff None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:35.784163 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb45752-91b0-4a3a-a21f-a335bff2b8ff None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:35.784447 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb45752-91b0-4a3a-a21f-a335bff2b8ff None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:35.785261 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb45752-91b0-4a3a-a21f-a335bff2b8ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:35.785576 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb45752-91b0-4a3a-a21f-a335bff2b8ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:35.787151 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb45752-91b0-4a3a-a21f-a335bff2b8ff None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:35.802417 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:35.805262 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 399/795] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:35.806960 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb45752-91b0-4a3a-a21f-a335bff2b8ff None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:35.820221 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b7fdaa-128e-4b0f-811c-aa2518f9da8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:35.821085 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7fdaa-128e-4b0f-811c-aa2518f9da8c None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:35.821368 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7fdaa-128e-4b0f-811c-aa2518f9da8c None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:35.821640 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7fdaa-128e-4b0f-811c-aa2518f9da8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:35.845022 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b7fdaa-128e-4b0f-811c-aa2518f9da8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:35.846540 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b7fdaa-128e-4b0f-811c-aa2518f9da8c None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:35.883743 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-b3b7fdaa-128e-4b0f-811c-aa2518f9da8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:35.883743 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:35.883743 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:35.883743 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:35.883743 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:35.883743 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:35.883743 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:35.883743 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:35.883743 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:35.883743 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:35.883743 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:35.883743 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:35.883743 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:35.883743 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:35.883743 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:35.884704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:35.884704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:35.884704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:35.884704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:35.884704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:35.884704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:35.884704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:35.884704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:35.884704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:35.884704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:35.884704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:35.884704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:35.884704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:35.884704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:35.884704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:35.884704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:35.884704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:35.885680 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:35.885680 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:35.885680 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:35.885680 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:35.886441 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 397/796] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:35.895995 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b7fdaa-128e-4b0f-811c-aa2518f9da8c None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:35.985216 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:35.988796 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb45752-91b0-4a3a-a21f-a335bff2b8ff None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:36.032956 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b7fdaa-128e-4b0f-811c-aa2518f9da8c None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:36.066849 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75cce00-0560-4bd5-b5e2-47faed8cc4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:36.067533 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cce00-0560-4bd5-b5e2-47faed8cc4e9 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:36.067815 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cce00-0560-4bd5-b5e2-47faed8cc4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:36.068089 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cce00-0560-4bd5-b5e2-47faed8cc4e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:36.068896 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75cce00-0560-4bd5-b5e2-47faed8cc4e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:36.069226 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75cce00-0560-4bd5-b5e2-47faed8cc4e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:36.070623 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75cce00-0560-4bd5-b5e2-47faed8cc4e9 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:36.084158 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 398/797] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:36.089980 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75cce00-0560-4bd5-b5e2-47faed8cc4e9 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:36.098227 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b72c6-4d04-4b39-bae1-57978a60ab55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:36.099053 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b72c6-4d04-4b39-bae1-57978a60ab55 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:36.099340 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b72c6-4d04-4b39-bae1-57978a60ab55 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:36.099620 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b72c6-4d04-4b39-bae1-57978a60ab55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:36.118180 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b72c6-4d04-4b39-bae1-57978a60ab55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:36.119741 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b72c6-4d04-4b39-bae1-57978a60ab55 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:36.149154 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-c23b72c6-4d04-4b39-bae1-57978a60ab55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:36.149154 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:36.149154 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:36.149154 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:36.149154 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:36.149154 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:36.149154 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:36.149154 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:36.149154 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:36.149154 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:36.149154 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:36.149154 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:36.149154 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:36.149154 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:36.149154 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:36.150128 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:36.150128 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:36.150128 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:36.150128 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:36.150128 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:36.150128 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:36.150128 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:36.150128 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:36.150128 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:36.150128 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:36.150128 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:36.150128 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:36.150128 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:36.150128 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:36.150128 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:36.150128 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:36.150128 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:36.151074 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:36.151074 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:36.151074 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:36.151074 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:36.151074 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:36.151074 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:36.151074 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:36.151074 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:36.157341 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 400/798] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:36.158351 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b72c6-4d04-4b39-bae1-57978a60ab55 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:36.209539 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:36.212193 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75cce00-0560-4bd5-b5e2-47faed8cc4e9 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:36.234923 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:36.251668 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a497807-d728-4441-8fac-ff4b5ef48a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:36.252400 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a497807-d728-4441-8fac-ff4b5ef48a31 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:36.252688 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a497807-d728-4441-8fac-ff4b5ef48a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:36.252980 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a497807-d728-4441-8fac-ff4b5ef48a31 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:36.261358 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a497807-d728-4441-8fac-ff4b5ef48a31 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:36.262832 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a497807-d728-4441-8fac-ff4b5ef48a31 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:36.278404 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-1a497807-d728-4441-8fac-ff4b5ef48a31 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 27 21:03:36.278404 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:36.278404 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:36.278404 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:36.278404 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:36.278404 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:36.278404 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:36.278404 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:36.278404 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:36.278404 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:36.278404 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:36.278404 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:36.278404 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:36.278404 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:36.278404 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:36.278404 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:36.279634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:36.279634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:36.279634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:36.279634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:36.279634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:36.279634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:36.279634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:36.279634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:36.279634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:36.279634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:36.279634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:36.279634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:36.279634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:36.279634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:36.279634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:36.279634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:36.279634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:36.279634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:36.280667 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:36.280667 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:36.280667 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:36.280667 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:36.280667 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:36.280667 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:36.280667 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:36.280667 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:36.280667 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 27 21:03:36.280667 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:36.280667 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 399/799] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 27 21:03:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:36.285301 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29f51c-6276-4531-8e37-d576520b1cdc None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:36.285604 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29f51c-6276-4531-8e37-d576520b1cdc None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:36.285885 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29f51c-6276-4531-8e37-d576520b1cdc None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:36.287446 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 400/800] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:36.288790 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b72c6-4d04-4b39-bae1-57978a60ab55 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:36.288955 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29f51c-6276-4531-8e37-d576520b1cdc None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:36.325995 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018a629-9068-4788-8e56-bfb8cb24a169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:36.326924 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018a629-9068-4788-8e56-bfb8cb24a169 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:36.327355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018a629-9068-4788-8e56-bfb8cb24a169 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:36.327777 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018a629-9068-4788-8e56-bfb8cb24a169 None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:36.328743 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4018a629-9068-4788-8e56-bfb8cb24a169 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:36.329230 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4018a629-9068-4788-8e56-bfb8cb24a169 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:36.330850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4018a629-9068-4788-8e56-bfb8cb24a169 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:36.348775 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 401/801] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:36.350478 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec845bd-5723-4869-97c8-46c621aaef2c None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:36.350946 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec845bd-5723-4869-97c8-46c621aaef2c None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:36.351719 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec845bd-5723-4869-97c8-46c621aaef2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:36.369725 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-7ec845bd-5723-4869-97c8-46c621aaef2c None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:36.371385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-7ec845bd-5723-4869-97c8-46c621aaef2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:36.402600 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29f51c-6276-4531-8e37-d576520b1cdc None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:36.440109 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c44a3-dc2e-4342-b91c-1975f0579d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:36.441032 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c44a3-dc2e-4342-b91c-1975f0579d09 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:36.441321 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c44a3-dc2e-4342-b91c-1975f0579d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:36.441599 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c44a3-dc2e-4342-b91c-1975f0579d09 None admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:36.442453 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241c44a3-dc2e-4342-b91c-1975f0579d09 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:36.442765 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241c44a3-dc2e-4342-b91c-1975f0579d09 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:36.444620 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241c44a3-dc2e-4342-b91c-1975f0579d09 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:36.446366 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 402/802] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:03:36.447405 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec845bd-5723-4869-97c8-46c621aaef2c None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:36.475626 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:36.479298 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 401/803] 10.222.0.13 () {56 vars in 1135 bytes} [Mon Jun 27 21:03:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:36.480296 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c44a3-dc2e-4342-b91c-1975f0579d09 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:36.544275 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:36.558879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec845bd-5723-4869-97c8-46c621aaef2c None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:36.598985 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ec389-83c2-45ab-90af-c9dde06c0b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:36.599804 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ec389-83c2-45ab-90af-c9dde06c0b54 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:36.600089 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ec389-83c2-45ab-90af-c9dde06c0b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:36.600366 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ec389-83c2-45ab-90af-c9dde06c0b54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:36.601584 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737ec389-83c2-45ab-90af-c9dde06c0b54 None admin] RBAC: Adding query filter params (role.id=d336dd7b746c4488b33289c00302e4d5, scope.project.id=334438c929c44e489e9a3b93e6d37231, user.id=1366958034d540b4ba2feb421a6afabc) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:36.601905 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737ec389-83c2-45ab-90af-c9dde06c0b54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:36.603358 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737ec389-83c2-45ab-90af-c9dde06c0b54 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:36.610623 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 403/804] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:03:36 2022] GET /identity/v3/role_assignments?user.id=1366958034d540b4ba2feb421a6afabc&scope.project.id=334438c929c44e489e9a3b93e6d37231&role.id=d336dd7b746c4488b33289c00302e4d5 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:36.611229 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c44a3-dc2e-4342-b91c-1975f0579d09 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:36.618032 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ec389-83c2-45ab-90af-c9dde06c0b54 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:36.664076 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eacc34-40b0-4564-8e42-9171e8cd89f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:36.665427 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eacc34-40b0-4564-8e42-9171e8cd89f2 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:36.665943 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eacc34-40b0-4564-8e42-9171e8cd89f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:36.666413 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eacc34-40b0-4564-8e42-9171e8cd89f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:36.684267 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eacc34-40b0-4564-8e42-9171e8cd89f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:36.695941 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-f3eacc34-40b0-4564-8e42-9171e8cd89f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:36.695941 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:36.695941 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:36.695941 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:36.695941 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:36.695941 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:36.695941 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:36.695941 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:36.695941 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:36.695941 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:36.695941 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:36.695941 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:36.695941 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:36.695941 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:36.695941 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:36.695941 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:36.697022 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:36.697022 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:36.697022 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:36.697022 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:36.697022 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:36.697022 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:36.697022 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:36.697022 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:36.697022 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:36.697022 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:36.697022 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:36.697022 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:36.697022 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:36.697022 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:36.697022 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:36.697022 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:36.697933 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:36.697933 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:36.697933 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:36.697933 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:36.697933 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:36.697933 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:36.697933 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:36.697933 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:36.697933 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:36.697933 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:36.697933 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:36.697933 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:36.707006 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 402/805] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 27 21:03:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:36.708709 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd6dc9-5201-4aa5-a84f-f0d41a6c8adf None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:36.709200 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd6dc9-5201-4aa5-a84f-f0d41a6c8adf None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:36.709565 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd6dc9-5201-4aa5-a84f-f0d41a6c8adf None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:36.715364 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 403/806] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:36.716596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd6dc9-5201-4aa5-a84f-f0d41a6c8adf None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:36.735317 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:36.751890 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ec389-83c2-45ab-90af-c9dde06c0b54 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:36.802027 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7456b4d2-3e1d-4fc4-bb19-54eb1356494b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:36.803386 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456b4d2-3e1d-4fc4-bb19-54eb1356494b None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:36.804395 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456b4d2-3e1d-4fc4-bb19-54eb1356494b None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:36.804688 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456b4d2-3e1d-4fc4-bb19-54eb1356494b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:36.821302 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7456b4d2-3e1d-4fc4-bb19-54eb1356494b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:36.824556 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7456b4d2-3e1d-4fc4-bb19-54eb1356494b None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:36.847260 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-7456b4d2-3e1d-4fc4-bb19-54eb1356494b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:36.847260 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:36.847260 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:36.847260 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:36.847260 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:36.847260 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:36.847260 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:36.847260 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:36.847260 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:36.847260 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:36.847260 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:36.847260 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:36.847260 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:36.847260 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:36.847260 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:36.848429 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:36.848429 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:36.848429 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:36.848429 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:36.848429 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:36.848429 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:36.848429 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:36.848429 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:36.848429 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:36.848429 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:36.848429 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:36.848429 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:36.848429 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:36.848429 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:36.848429 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:36.848429 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:36.848429 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:36.849516 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:36.849516 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:36.849516 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:36.849516 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:36.849774 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:36.851529 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 404/807] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:36.853194 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5100d9d-c2dd-424e-8f62-3f5180fe6412 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:36.853620 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5100d9d-c2dd-424e-8f62-3f5180fe6412 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:36.854118 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5100d9d-c2dd-424e-8f62-3f5180fe6412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:36.855041 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd6dc9-5201-4aa5-a84f-f0d41a6c8adf None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:36.872019 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-e5100d9d-c2dd-424e-8f62-3f5180fe6412 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:36.873272 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-e5100d9d-c2dd-424e-8f62-3f5180fe6412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:36.903448 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4c8b32-2068-4aa7-a4b2-e59888e39628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:36.904432 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c8b32-2068-4aa7-a4b2-e59888e39628 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:36.904877 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c8b32-2068-4aa7-a4b2-e59888e39628 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:36.905403 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c8b32-2068-4aa7-a4b2-e59888e39628 None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:36.906409 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4c8b32-2068-4aa7-a4b2-e59888e39628 None admin] RBAC: Adding query filter params (name=service) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:36.906876 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4c8b32-2068-4aa7-a4b2-e59888e39628 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:36.908690 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4c8b32-2068-4aa7-a4b2-e59888e39628 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:36.917872 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 404/808] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 27 21:03:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:36.919365 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae83d8-f189-4c13-9351-d60d658a44e6 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:36.919933 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae83d8-f189-4c13-9351-d60d658a44e6 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:36.920207 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae83d8-f189-4c13-9351-d60d658a44e6 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:36.921653 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 405/809] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:36.922136 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ae83d8-f189-4c13-9351-d60d658a44e6 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:36.948355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 405/810] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:03:36.949556 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5100d9d-c2dd-424e-8f62-3f5180fe6412 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:37.040139 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:37.073571 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5100d9d-c2dd-424e-8f62-3f5180fe6412 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:37.074358 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ae83d8-f189-4c13-9351-d60d658a44e6 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:37.111151 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957de00d-d778-4aba-a7b1-70d8c580e775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:37.111476 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9470a3e6-0fe0-4d10-9d5c-0d2ecd19bda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:37.112257 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957de00d-d778-4aba-a7b1-70d8c580e775 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:37.112375 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470a3e6-0fe0-4d10-9d5c-0d2ecd19bda8 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:37.112542 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470a3e6-0fe0-4d10-9d5c-0d2ecd19bda8 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:37.112817 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470a3e6-0fe0-4d10-9d5c-0d2ecd19bda8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:37.113076 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957de00d-d778-4aba-a7b1-70d8c580e775 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:37.113747 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957de00d-d778-4aba-a7b1-70d8c580e775 None admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:37.114675 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957de00d-d778-4aba-a7b1-70d8c580e775 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:37.114980 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957de00d-d778-4aba-a7b1-70d8c580e775 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:37.116444 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957de00d-d778-4aba-a7b1-70d8c580e775 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:37.123499 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9470a3e6-0fe0-4d10-9d5c-0d2ecd19bda8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:37.124983 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9470a3e6-0fe0-4d10-9d5c-0d2ecd19bda8 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:37.134482 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 406/811] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:37.136314 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab512be-68de-46f0-b151-9e13a1bcf93b None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:37.136606 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab512be-68de-46f0-b151-9e13a1bcf93b None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:37.136883 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab512be-68de-46f0-b151-9e13a1bcf93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:37.143017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-9470a3e6-0fe0-4d10-9d5c-0d2ecd19bda8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:37.143017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:37.143017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:37.143017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:37.143017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:37.143017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:37.143017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:37.143017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:37.143017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:37.143017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:37.143017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:37.143017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:37.143017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:37.143017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:37.143017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:37.143977 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:37.143977 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:37.143977 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:37.143977 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:37.143977 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:37.143977 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:37.143977 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:37.143977 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:37.143977 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:37.143977 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:37.143977 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:37.143977 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:37.143977 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:37.143977 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:37.143977 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:37.143977 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:37.143977 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:37.144870 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:37.144870 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:37.144870 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:37.144870 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:37.144870 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 406/812] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:37.145169 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9470a3e6-0fe0-4d10-9d5c-0d2ecd19bda8 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:37.160704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-9ab512be-68de-46f0-b151-9e13a1bcf93b None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:37.161989 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-9ab512be-68de-46f0-b151-9e13a1bcf93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:37.179102 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:37.240127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 407/813] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:03:37.241009 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab512be-68de-46f0-b151-9e13a1bcf93b None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:37.258557 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9470a3e6-0fe0-4d10-9d5c-0d2ecd19bda8 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:37.284258 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:37.296041 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc12bed-6292-4813-8cbe-678b760d0b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:37.296834 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc12bed-6292-4813-8cbe-678b760d0b99 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:37.297137 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc12bed-6292-4813-8cbe-678b760d0b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:37.297413 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc12bed-6292-4813-8cbe-678b760d0b99 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:37.298183 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc12bed-6292-4813-8cbe-678b760d0b99 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:37.299740 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc12bed-6292-4813-8cbe-678b760d0b99 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:37.312638 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-5dc12bed-6292-4813-8cbe-678b760d0b99 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 21:03:37.312638 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:37.312638 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:03:37.312638 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:03:37.312638 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:03:37.312638 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:03:37.312638 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:03:37.312638 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:37.312638 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 21:03:37.312638 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:37.312638 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:03:37.312638 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:37.312638 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:37.312638 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:37.312638 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:37.312638 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:37.313787 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:37.313787 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:37.313787 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:37.313787 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:37.313787 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:37.313787 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:37.313787 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:37.313787 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:03:37.313787 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:03:37.313787 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:03:37.313787 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:03:37.313787 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:37.313787 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:37.313787 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:37.313787 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:37.313787 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:37.314616 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:37.314616 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:37.314616 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:37.314616 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:37.314616 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:37.314616 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:37.314616 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:37.314616 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:37.314616 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:37.314616 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:37.314616 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:37.314616 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:03:37.314616 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:37.314616 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 27 21:03:37.314616 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:37.314616 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 407/814] 10.222.0.13 () {56 vars in 1129 bytes} [Mon Jun 27 21:03:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:37.315543 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc12bed-6292-4813-8cbe-678b760d0b99 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:37.360514 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab512be-68de-46f0-b151-9e13a1bcf93b None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:37.443248 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc12bed-6292-4813-8cbe-678b760d0b99 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:37.496630 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c84432-17d7-46e0-ba4f-a849e49e487c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:37.497707 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c84432-17d7-46e0-ba4f-a849e49e487c None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:37.498144 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c84432-17d7-46e0-ba4f-a849e49e487c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:37.498596 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c84432-17d7-46e0-ba4f-a849e49e487c None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:37.499608 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c84432-17d7-46e0-ba4f-a849e49e487c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:37.500055 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c84432-17d7-46e0-ba4f-a849e49e487c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:37.502002 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c84432-17d7-46e0-ba4f-a849e49e487c None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:37.515517 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 408/815] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:37.516363 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c84432-17d7-46e0-ba4f-a849e49e487c None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:37.524215 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c40fe9-c236-44b0-8b45-3cc9e43e3d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:37.525070 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c40fe9-c236-44b0-8b45-3cc9e43e3d58 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:37.525355 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c40fe9-c236-44b0-8b45-3cc9e43e3d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:37.525628 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c40fe9-c236-44b0-8b45-3cc9e43e3d58 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:37.532890 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c40fe9-c236-44b0-8b45-3cc9e43e3d58 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:37.534372 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c40fe9-c236-44b0-8b45-3cc9e43e3d58 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:37.549188 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-f5c40fe9-c236-44b0-8b45-3cc9e43e3d58 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:37.549188 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:37.549188 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:37.549188 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:37.549188 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:37.549188 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:37.549188 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:37.549188 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:37.549188 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:37.549188 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:37.549188 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:37.549188 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:37.549188 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:37.549188 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:37.549188 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:37.550255 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:37.550255 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:37.550255 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:37.550255 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:37.550255 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:37.550255 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:37.550255 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:37.550255 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:37.550255 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:37.550255 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:37.550255 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:37.550255 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:37.550255 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:37.550255 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:37.550255 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:37.550255 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:37.550255 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:37.551332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:37.551332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:37.551332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:37.551332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:37.551332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:37.551332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:37.551332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:37.551332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:37.555315 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 408/816] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:37.556154 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c40fe9-c236-44b0-8b45-3cc9e43e3d58 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:37.569193 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:37.573924 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:37.627401 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c84432-17d7-46e0-ba4f-a849e49e487c None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:37.662129 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8570b93-4dbc-4975-828f-0a14b86ef360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:37.662503 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c40fe9-c236-44b0-8b45-3cc9e43e3d58 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:37.662983 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8570b93-4dbc-4975-828f-0a14b86ef360 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:37.663259 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8570b93-4dbc-4975-828f-0a14b86ef360 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:37.663525 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8570b93-4dbc-4975-828f-0a14b86ef360 None admin] PATH_INFO: `/v3/services/network` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:37.664258 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8570b93-4dbc-4975-828f-0a14b86ef360 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:37.665653 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8570b93-4dbc-4975-828f-0a14b86ef360 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:37.677435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-b8570b93-4dbc-4975-828f-0a14b86ef360 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 21:03:37.677435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:37.677435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:03:37.677435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:03:37.677435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:03:37.677435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:03:37.677435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:03:37.677435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:37.677435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 21:03:37.677435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:37.677435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:03:37.677435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:37.677435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:37.677435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:37.677435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:37.677435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:37.677435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:37.678442 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:37.678442 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:37.678442 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:37.678442 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:37.678442 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:37.678442 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:37.678442 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:03:37.678442 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:03:37.678442 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:03:37.678442 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:03:37.678442 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:37.678442 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:37.678442 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:37.678442 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:37.678442 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:37.678442 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:37.679269 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:37.679269 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:37.679269 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:37.679269 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:37.679269 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:37.679269 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:37.679269 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:37.679269 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:37.679269 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:37.679269 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:37.679269 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:03:37.679269 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:37.679269 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 27 21:03:37.679269 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:37.679269 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 409/817] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:37 2022] GET /identity/v3/services/network => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:37.680063 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8570b93-4dbc-4975-828f-0a14b86ef360 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:37.706894 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5dea8-0d6f-49bf-a36f-41a185bd55be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:37.707566 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5dea8-0d6f-49bf-a36f-41a185bd55be None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:37.707841 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5dea8-0d6f-49bf-a36f-41a185bd55be None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:37.708115 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5dea8-0d6f-49bf-a36f-41a185bd55be None admin] PATH_INFO: `/v3/services` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:37.709303 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c5dea8-0d6f-49bf-a36f-41a185bd55be None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:37.709606 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c5dea8-0d6f-49bf-a36f-41a185bd55be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:37.710970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c5dea8-0d6f-49bf-a36f-41a185bd55be None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:37.715552 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 409/818] 10.222.0.13 () {56 vars in 1124 bytes} [Mon Jun 27 21:03:37 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:37.716320 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5dea8-0d6f-49bf-a36f-41a185bd55be None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:37.802035 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:37.807057 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8570b93-4dbc-4975-828f-0a14b86ef360 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:37.823601 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:37.836015 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5dea8-0d6f-49bf-a36f-41a185bd55be None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:37.850615 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b646b8d-7c90-4e76-ae38-a981e9f69be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:37.852188 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b646b8d-7c90-4e76-ae38-a981e9f69be9 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:37.852645 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b646b8d-7c90-4e76-ae38-a981e9f69be9 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:37.853118 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b646b8d-7c90-4e76-ae38-a981e9f69be9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:37.862793 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b646b8d-7c90-4e76-ae38-a981e9f69be9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:37.864747 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b646b8d-7c90-4e76-ae38-a981e9f69be9 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:37.878221 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f1fdc-709f-4666-a70f-b7afaede8c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:37.879184 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f1fdc-709f-4666-a70f-b7afaede8c25 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:37.879640 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f1fdc-709f-4666-a70f-b7afaede8c25 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:37.880082 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f1fdc-709f-4666-a70f-b7afaede8c25 None admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:37.881099 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-5b646b8d-7c90-4e76-ae38-a981e9f69be9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 21:03:37.881099 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:37.881099 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:37.881099 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:37.881099 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:37.881099 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:37.881099 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:37.881099 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:37.881099 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:37.881099 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:37.881099 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:37.881099 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:37.881099 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:37.881099 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:37.881099 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:37.881099 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:37.882007 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:37.882007 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:37.882007 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:37.882007 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:37.882007 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:37.882007 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:37.882007 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:37.882007 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:37.882007 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:37.882007 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:37.882007 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:37.882007 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:37.882007 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:37.882007 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:37.882007 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:37.882007 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:37.882007 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:37.882007 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:37.882954 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:37.882954 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:37.882954 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:37.882954 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:37.882954 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:37.882954 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:37.882954 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:37.882954 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:37.882954 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 21:03:37.882954 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:37.882954 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 410/819] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Jun 27 21:03:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:37.883540 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b646b8d-7c90-4e76-ae38-a981e9f69be9 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:37.883975 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87f1fdc-709f-4666-a70f-b7afaede8c25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:37.884314 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87f1fdc-709f-4666-a70f-b7afaede8c25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:37.885866 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87f1fdc-709f-4666-a70f-b7afaede8c25 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:37.901649 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 410/820] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:37.902579 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f1fdc-709f-4666-a70f-b7afaede8c25 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:38.003005 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b646b8d-7c90-4e76-ae38-a981e9f69be9 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:38.015969 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f1fdc-709f-4666-a70f-b7afaede8c25 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:38.043700 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce35b3ee-5339-4fa0-9625-40fb177c5589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:38.044418 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35b3ee-5339-4fa0-9625-40fb177c5589 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:38.044594 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35b3ee-5339-4fa0-9625-40fb177c5589 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:38.044861 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35b3ee-5339-4fa0-9625-40fb177c5589 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:38.045583 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce35b3ee-5339-4fa0-9625-40fb177c5589 None admin] RBAC: Adding query filter params (name=network) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:38.045890 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce35b3ee-5339-4fa0-9625-40fb177c5589 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:38.047257 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce35b3ee-5339-4fa0-9625-40fb177c5589 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:38.052162 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 411/821] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Jun 27 21:03:37 2022] GET /identity/v3/services?name=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:38.052851 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce35b3ee-5339-4fa0-9625-40fb177c5589 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:38.059768 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285af43f-f96a-4fdf-a65a-44f16187b41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:38.060550 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285af43f-f96a-4fdf-a65a-44f16187b41e None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:38.060899 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285af43f-f96a-4fdf-a65a-44f16187b41e None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:38.061268 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285af43f-f96a-4fdf-a65a-44f16187b41e None admin] PATH_INFO: `/v3/services` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:38.062360 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285af43f-f96a-4fdf-a65a-44f16187b41e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:38.062360 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285af43f-f96a-4fdf-a65a-44f16187b41e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:38.063854 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285af43f-f96a-4fdf-a65a-44f16187b41e None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:38.065647 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:38.075492 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 411/822] 10.222.0.13 () {56 vars in 1124 bytes} [Mon Jun 27 21:03:37 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:38.076963 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285af43f-f96a-4fdf-a65a-44f16187b41e None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:38.094676 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:38.170956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce35b3ee-5339-4fa0-9625-40fb177c5589 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:38.188370 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285af43f-f96a-4fdf-a65a-44f16187b41e None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:38.207705 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b1c343-4fe3-49c6-a16f-51f1c27d6247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:38.208682 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b1c343-4fe3-49c6-a16f-51f1c27d6247 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:38.209143 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b1c343-4fe3-49c6-a16f-51f1c27d6247 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:38.209579 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b1c343-4fe3-49c6-a16f-51f1c27d6247 None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:38.210653 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b1c343-4fe3-49c6-a16f-51f1c27d6247 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:38.211298 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b1c343-4fe3-49c6-a16f-51f1c27d6247 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:38.213329 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b1c343-4fe3-49c6-a16f-51f1c27d6247 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:38.239847 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 412/823] 10.222.0.13 () {56 vars in 1149 bytes} [Mon Jun 27 21:03:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:38.239847 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b1c343-4fe3-49c6-a16f-51f1c27d6247 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:38.249646 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d9a29-8612-4da8-98fe-7b1ee9ba130b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:38.250557 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d9a29-8612-4da8-98fe-7b1ee9ba130b None admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:38.250838 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d9a29-8612-4da8-98fe-7b1ee9ba130b None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:38.251108 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d9a29-8612-4da8-98fe-7b1ee9ba130b None admin] PATH_INFO: `/v3/projects/334438c929c44e489e9a3b93e6d37231/users/8bed6b231ee647d8a350da055d8e59d7/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:38.253126 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912d9a29-8612-4da8-98fe-7b1ee9ba130b None admin] RBAC: Authorizing `identity:create_grant(project_id=334438c929c44e489e9a3b93e6d37231, user_id=8bed6b231ee647d8a350da055d8e59d7, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:38.254559 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912d9a29-8612-4da8-98fe-7b1ee9ba130b None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:38.276961 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 412/824] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 27 21:03:38 2022] PUT /identity/v3/projects/334438c929c44e489e9a3b93e6d37231/users/8bed6b231ee647d8a350da055d8e59d7/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:03:38.276961 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d9a29-8612-4da8-98fe-7b1ee9ba130b None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:38.357227 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:38.366162 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:38.366505 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b1c343-4fe3-49c6-a16f-51f1c27d6247 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:38.417507 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a3771-0608-4a95-b24f-f8bae4b33005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:38.418764 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a3771-0608-4a95-b24f-f8bae4b33005 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:38.419043 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a3771-0608-4a95-b24f-f8bae4b33005 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:38.419317 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a3771-0608-4a95-b24f-f8bae4b33005 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:38.420367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25a3771-0608-4a95-b24f-f8bae4b33005 None admin] RBAC: Adding query filter params (type=network) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:38.420808 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25a3771-0608-4a95-b24f-f8bae4b33005 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:38.422847 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25a3771-0608-4a95-b24f-f8bae4b33005 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:38.430509 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 413/825] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Jun 27 21:03:38 2022] GET /identity/v3/services?type=network => generated 350 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:38.431488 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a3771-0608-4a95-b24f-f8bae4b33005 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:38.436161 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d9a29-8612-4da8-98fe-7b1ee9ba130b None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:38.482083 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1cb0c5-a353-43c8-accf-eef57224f88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:38.483152 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1cb0c5-a353-43c8-accf-eef57224f88e None admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:38.483609 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1cb0c5-a353-43c8-accf-eef57224f88e None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:38.484061 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1cb0c5-a353-43c8-accf-eef57224f88e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:38.485395 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1cb0c5-a353-43c8-accf-eef57224f88e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:38.487834 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1cb0c5-a353-43c8-accf-eef57224f88e None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:38.526170 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 413/826] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Jun 27 21:03:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:03:38.527095 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1cb0c5-a353-43c8-accf-eef57224f88e None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:38.594219 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a3771-0608-4a95-b24f-f8bae4b33005 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:38.680038 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf3010-eb9e-40eb-87e6-b9c37becf8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:38.680038 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf3010-eb9e-40eb-87e6-b9c37becf8c7 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:38.680038 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf3010-eb9e-40eb-87e6-b9c37becf8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:38.680038 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf3010-eb9e-40eb-87e6-b9c37becf8c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:38.696384 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf3010-eb9e-40eb-87e6-b9c37becf8c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:38.698514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf3010-eb9e-40eb-87e6-b9c37becf8c7 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:38.721498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-bcdf3010-eb9e-40eb-87e6-b9c37becf8c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:38.721498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:38.721498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:38.721498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:38.721498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:38.721498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:38.721498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:38.721498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:38.721498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:38.721498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:38.721498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:38.721498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:38.721498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:38.721498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:38.721498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:38.722469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:38.722469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:38.722469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:38.722469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:38.722469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:38.722469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:38.722469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:38.722469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:38.722469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:38.722469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:38.722469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:38.722469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:38.722469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:38.722469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:38.722469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:38.722469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:38.722469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:38.723395 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:38.723395 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:38.723395 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:38.723395 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:38.727725 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 414/827] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:38.729259 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf3010-eb9e-40eb-87e6-b9c37becf8c7 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:38.732373 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1cb0c5-a353-43c8-accf-eef57224f88e None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:38.741059 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:38.750814 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:38.776624 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b39737a-6a21-4ba4-abbf-d03132f22a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:38.777407 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39737a-6a21-4ba4-abbf-d03132f22a29 None admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:38.777849 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39737a-6a21-4ba4-abbf-d03132f22a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:38.778126 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39737a-6a21-4ba4-abbf-d03132f22a29 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:38.779766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b39737a-6a21-4ba4-abbf-d03132f22a29 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:38.781228 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b39737a-6a21-4ba4-abbf-d03132f22a29 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:38.812806 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 414/828] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Jun 27 21:03:38 2022] POST /identity/v3/endpoints => generated 327 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:03:38.821050 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841c657-e6e1-475a-99cf-48c4deba09ac None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:38.821050 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841c657-e6e1-475a-99cf-48c4deba09ac None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:38.821050 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841c657-e6e1-475a-99cf-48c4deba09ac None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:38.821050 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 415/829] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:38.822678 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d97a1-1642-48ef-8e4d-c108775f9ecb None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:38.823079 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d97a1-1642-48ef-8e4d-c108775f9ecb None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:38.823363 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d97a1-1642-48ef-8e4d-c108775f9ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:38.843496 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-f23d97a1-1642-48ef-8e4d-c108775f9ecb None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:38.844796 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-f23d97a1-1642-48ef-8e4d-c108775f9ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:38.872228 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf3010-eb9e-40eb-87e6-b9c37becf8c7 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:38.928646 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b49f13-95ac-4597-938a-29172be7679b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:38.929531 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b49f13-95ac-4597-938a-29172be7679b None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:38.929814 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b49f13-95ac-4597-938a-29172be7679b None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:38.930143 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b49f13-95ac-4597-938a-29172be7679b None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:38.930833 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b49f13-95ac-4597-938a-29172be7679b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:38.931136 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b49f13-95ac-4597-938a-29172be7679b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:38.932940 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b49f13-95ac-4597-938a-29172be7679b None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:38.938722 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 416/830] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:38 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:03:38.950877 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 415/831] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:38.952419 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d97a1-1642-48ef-8e4d-c108775f9ecb None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:38.977148 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b49f13-95ac-4597-938a-29172be7679b None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:39.057072 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:39.125699 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b49f13-95ac-4597-938a-29172be7679b None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:39.133081 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:39.139684 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d97a1-1642-48ef-8e4d-c108775f9ecb None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:39.164616 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61b41a1-5986-4d83-83c7-ee7b853ec70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:39.165848 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b41a1-5986-4d83-83c7-ee7b853ec70f None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:39.166145 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b41a1-5986-4d83-83c7-ee7b853ec70f None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:39.166587 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b41a1-5986-4d83-83c7-ee7b853ec70f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:39.168770 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61b41a1-5986-4d83-83c7-ee7b853ec70f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:39.170095 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61b41a1-5986-4d83-83c7-ee7b853ec70f None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:39.189576 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-b61b41a1-5986-4d83-83c7-ee7b853ec70f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 21:03:39.189576 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:39.189576 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:03:39.189576 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:03:39.189576 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:03:39.189576 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:03:39.189576 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:03:39.189576 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:39.189576 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 21:03:39.189576 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:39.189576 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:03:39.189576 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:39.189576 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:39.189576 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:39.189576 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:39.189576 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:39.191620 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:39.191620 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:39.191620 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:39.191620 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:39.191620 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:39.191620 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:39.191620 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:39.191620 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:03:39.191620 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:03:39.191620 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:03:39.191620 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:03:39.191620 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:39.191620 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:39.191620 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:39.191620 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:39.191620 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:39.193411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:39.193411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:39.193411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:39.193411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:39.193411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:39.193411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:39.193411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:39.193411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:39.193411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:39.193411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:39.193411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:39.193411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:03:39.193411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:39.193411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 21:03:39.193411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:39.193411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 416/832] 10.222.0.13 () {56 vars in 1132 bytes} [Mon Jun 27 21:03:38 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:39.198192 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61b41a1-5986-4d83-83c7-ee7b853ec70f None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:39.202135 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be093a0-c8e6-40ed-bf83-38406ed24774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:39.202972 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be093a0-c8e6-40ed-bf83-38406ed24774 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:39.203243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be093a0-c8e6-40ed-bf83-38406ed24774 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:39.203518 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be093a0-c8e6-40ed-bf83-38406ed24774 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:39.218712 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be093a0-c8e6-40ed-bf83-38406ed24774 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:39.220638 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be093a0-c8e6-40ed-bf83-38406ed24774 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:39.236961 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-6be093a0-c8e6-40ed-bf83-38406ed24774 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:39.236961 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:39.236961 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:39.236961 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:39.236961 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:39.236961 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:39.236961 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:39.236961 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:39.236961 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:39.236961 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:39.236961 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:39.236961 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:39.236961 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:39.236961 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:39.236961 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:39.237946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:39.237946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:39.237946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:39.237946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:39.237946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:39.237946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:39.237946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:39.237946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:39.237946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:39.237946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:39.237946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:39.237946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:39.237946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:39.237946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:39.237946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:39.237946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:39.237946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:39.238836 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:39.238836 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:39.238836 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:39.238836 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:39.238836 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:39.238836 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:39.238836 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:39.238836 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:39.245058 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 417/833] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:39.247933 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be093a0-c8e6-40ed-bf83-38406ed24774 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:39.325154 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61b41a1-5986-4d83-83c7-ee7b853ec70f None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:39.353783 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:39.357913 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be093a0-c8e6-40ed-bf83-38406ed24774 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:39.363575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f231a930-6daa-42c0-93ba-3c7ed531ca4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:39.364377 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231a930-6daa-42c0-93ba-3c7ed531ca4b None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:39.364757 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231a930-6daa-42c0-93ba-3c7ed531ca4b None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:39.365160 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231a930-6daa-42c0-93ba-3c7ed531ca4b None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:39.365984 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f231a930-6daa-42c0-93ba-3c7ed531ca4b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:39.366449 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f231a930-6daa-42c0-93ba-3c7ed531ca4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:39.368063 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f231a930-6daa-42c0-93ba-3c7ed531ca4b None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:39.375503 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 417/834] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Jun 27 21:03:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:39.386555 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f231a930-6daa-42c0-93ba-3c7ed531ca4b None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:39.410840 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb1d335-2d9b-49d5-8d55-7072a9c1ee13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:39.412248 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1d335-2d9b-49d5-8d55-7072a9c1ee13 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:39.412948 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1d335-2d9b-49d5-8d55-7072a9c1ee13 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:39.413642 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb1d335-2d9b-49d5-8d55-7072a9c1ee13 None admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:39.415374 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb1d335-2d9b-49d5-8d55-7072a9c1ee13 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:39.416117 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb1d335-2d9b-49d5-8d55-7072a9c1ee13 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:39.420035 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb1d335-2d9b-49d5-8d55-7072a9c1ee13 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:39.443001 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 418/835] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:39.445012 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb1d335-2d9b-49d5-8d55-7072a9c1ee13 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:39.478202 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:39.517514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f231a930-6daa-42c0-93ba-3c7ed531ca4b None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:39.552577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1097229-4527-433b-9d0f-07382101b00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:39.553577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1097229-4527-433b-9d0f-07382101b00d None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:39.554044 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1097229-4527-433b-9d0f-07382101b00d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:39.554451 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1097229-4527-433b-9d0f-07382101b00d None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:39.555291 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1097229-4527-433b-9d0f-07382101b00d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:39.555750 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1097229-4527-433b-9d0f-07382101b00d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:39.557958 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1097229-4527-433b-9d0f-07382101b00d None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:39.563254 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 418/836] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Jun 27 21:03:39 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:39.587315 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb1d335-2d9b-49d5-8d55-7072a9c1ee13 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:39.628197 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a1ba5a-a060-4b52-84e1-366f417fd9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:39.629494 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1ba5a-a060-4b52-84e1-366f417fd9df None admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:39.629975 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1ba5a-a060-4b52-84e1-366f417fd9df None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:39.630385 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1ba5a-a060-4b52-84e1-366f417fd9df None admin] PATH_INFO: `/v3/projects/334438c929c44e489e9a3b93e6d37231/users/f67110d591a94537a3df6a2d835370f9/roles/709aef6b85524eb2a8e9986a2bd9525b` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:39.648770 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a1ba5a-a060-4b52-84e1-366f417fd9df None admin] RBAC: Authorizing `identity:create_grant(project_id=334438c929c44e489e9a3b93e6d37231, user_id=f67110d591a94537a3df6a2d835370f9, role_id=709aef6b85524eb2a8e9986a2bd9525b)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:39.650495 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a1ba5a-a060-4b52-84e1-366f417fd9df None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:39.673253 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 419/837] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 27 21:03:39 2022] PUT /identity/v3/projects/334438c929c44e489e9a3b93e6d37231/users/f67110d591a94537a3df6a2d835370f9/roles/709aef6b85524eb2a8e9986a2bd9525b => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:03:40.682341 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a7ccd-f2f7-4cf6-b979-dabeaad6b5a3 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:40.683582 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a7ccd-f2f7-4cf6-b979-dabeaad6b5a3 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:40.683582 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a7ccd-f2f7-4cf6-b979-dabeaad6b5a3 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:40.684505 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 419/838] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:40.690167 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e56d1a-1a8c-4f7a-928a-efe5d7a4bfb9 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:40.690612 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e56d1a-1a8c-4f7a-928a-efe5d7a4bfb9 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:40.690794 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e56d1a-1a8c-4f7a-928a-efe5d7a4bfb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:40.711353 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-c1e56d1a-1a8c-4f7a-928a-efe5d7a4bfb9 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:40.712567 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-c1e56d1a-1a8c-4f7a-928a-efe5d7a4bfb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:40.781378 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9032a-f74e-4e38-a951-5a197c9315e1 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:40.781736 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9032a-f74e-4e38-a951-5a197c9315e1 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:40.781929 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9032a-f74e-4e38-a951-5a197c9315e1 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:40.783153 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:40.783605 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 420/839] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:40.789227 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4051d887-bcea-43ee-a691-bea72468c053 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:40.789522 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4051d887-bcea-43ee-a691-bea72468c053 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:40.789792 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4051d887-bcea-43ee-a691-bea72468c053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:40.797244 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 420/840] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:03:40.806919 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-4051d887-bcea-43ee-a691-bea72468c053 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:40.808336 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-4051d887-bcea-43ee-a691-bea72468c053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:40.816954 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e56d1a-1a8c-4f7a-928a-efe5d7a4bfb9 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:40.866206 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:40.891683 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 421/841] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:03:40.915340 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4051d887-bcea-43ee-a691-bea72468c053 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:40.938206 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e56d1a-1a8c-4f7a-928a-efe5d7a4bfb9 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:40.986691 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6612595-f664-4aae-b60b-46f1569544b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:40.988035 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6612595-f664-4aae-b60b-46f1569544b0 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:40.988908 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6612595-f664-4aae-b60b-46f1569544b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:40.989363 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6612595-f664-4aae-b60b-46f1569544b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:41.003373 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6612595-f664-4aae-b60b-46f1569544b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:41.013350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-d6612595-f664-4aae-b60b-46f1569544b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:03:41.013350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:41.013350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:41.013350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:41.013350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:41.013350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:41.013350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:41.013350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:41.013350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:41.013350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:41.013350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:41.013350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:41.013350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:41.013350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:41.013350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:41.013350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:41.014497 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:41.014497 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:41.014497 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:41.014497 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:41.014497 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:41.014497 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:41.014497 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:41.014497 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:41.014497 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:41.014497 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:41.014497 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:41.014497 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:41.014497 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:41.014497 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:41.014497 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:41.014497 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:41.015546 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:41.015546 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:41.015546 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:41.015546 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:41.015546 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:41.015546 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:41.015546 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:41.015546 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:41.015546 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:41.015546 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:41.015546 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:03:41.015546 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:41.018433 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 421/842] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 27 21:03:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:41.024487 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6612595-f664-4aae-b60b-46f1569544b0 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:41.026094 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4051d887-bcea-43ee-a691-bea72468c053 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:41.067051 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b26d831-7bc9-4afc-b8cb-777bc200e13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:41.068429 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26d831-7bc9-4afc-b8cb-777bc200e13b None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:41.068806 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26d831-7bc9-4afc-b8cb-777bc200e13b None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:41.069334 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26d831-7bc9-4afc-b8cb-777bc200e13b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:41.070218 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b26d831-7bc9-4afc-b8cb-777bc200e13b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:41.072379 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b26d831-7bc9-4afc-b8cb-777bc200e13b None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:41.094984 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-0b26d831-7bc9-4afc-b8cb-777bc200e13b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 21:03:41.094984 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:41.094984 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:03:41.094984 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:03:41.094984 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:03:41.094984 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:03:41.094984 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:03:41.094984 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:41.094984 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 21:03:41.094984 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:41.094984 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:03:41.094984 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:41.094984 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:41.094984 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:41.094984 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:41.094984 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:41.094984 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:41.096133 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:41.096133 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:41.096133 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:41.096133 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:41.096133 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:41.096133 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:41.096133 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:03:41.096133 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:03:41.096133 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:03:41.096133 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:03:41.096133 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:41.096133 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:41.096133 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:41.096133 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:41.096133 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:41.096133 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:41.097001 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:41.097001 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:41.097001 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:41.097001 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:41.097001 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:41.097001 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:41.097001 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:41.097001 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:41.097001 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:41.097001 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:41.097001 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:03:41.097001 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:41.097001 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 21:03:41.097001 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:41.097001 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 422/843] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 27 21:03:40 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:41.104053 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b26d831-7bc9-4afc-b8cb-777bc200e13b None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:41.111097 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:41.173096 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6612595-f664-4aae-b60b-46f1569544b0 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:41.181519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:41.221828 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a7b590-23d0-4238-a241-cd652581826b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:41.223146 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7b590-23d0-4238-a241-cd652581826b None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:41.223578 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7b590-23d0-4238-a241-cd652581826b None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:41.223998 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7b590-23d0-4238-a241-cd652581826b None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:41.224899 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a7b590-23d0-4238-a241-cd652581826b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:41.225388 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a7b590-23d0-4238-a241-cd652581826b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:41.228162 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a7b590-23d0-4238-a241-cd652581826b None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:41.238948 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b26d831-7bc9-4afc-b8cb-777bc200e13b None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:41.245100 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 422/844] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:03:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:41.246939 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a7b590-23d0-4238-a241-cd652581826b None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:41.287389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a6b18-7176-4e25-9f08-9b9f5e8ef356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:41.288471 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a6b18-7176-4e25-9f08-9b9f5e8ef356 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:41.289204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a6b18-7176-4e25-9f08-9b9f5e8ef356 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:41.290542 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6a6b18-7176-4e25-9f08-9b9f5e8ef356 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:41.291505 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a6b18-7176-4e25-9f08-9b9f5e8ef356 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:41.291996 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a6b18-7176-4e25-9f08-9b9f5e8ef356 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:41.293610 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6a6b18-7176-4e25-9f08-9b9f5e8ef356 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:41.300901 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 423/845] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 27 21:03:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:41.306671 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a6b18-7176-4e25-9f08-9b9f5e8ef356 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:41.373887 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a7b590-23d0-4238-a241-cd652581826b None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:41.405914 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:41.419056 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6a6b18-7176-4e25-9f08-9b9f5e8ef356 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:41.424421 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f45619-e381-434c-8d4a-35f9c53532ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:41.425545 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f45619-e381-434c-8d4a-35f9c53532ce None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:41.426051 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f45619-e381-434c-8d4a-35f9c53532ce None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:41.426523 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f45619-e381-434c-8d4a-35f9c53532ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:41.433853 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:41.444779 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f45619-e381-434c-8d4a-35f9c53532ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:41.446667 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f45619-e381-434c-8d4a-35f9c53532ce None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:41.468673 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1bae3b-008b-4e9e-890b-ab75060574aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:41.469399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1bae3b-008b-4e9e-890b-ab75060574aa None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:41.469677 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1bae3b-008b-4e9e-890b-ab75060574aa None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:41.469951 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1bae3b-008b-4e9e-890b-ab75060574aa None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:41.470672 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1bae3b-008b-4e9e-890b-ab75060574aa None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:41.470977 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1bae3b-008b-4e9e-890b-ab75060574aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:41.472422 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1bae3b-008b-4e9e-890b-ab75060574aa None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:41.474142 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-48f45619-e381-434c-8d4a-35f9c53532ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:41.474142 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:41.474142 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:41.474142 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:41.474142 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:41.474142 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:41.474142 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:41.474142 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:41.474142 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:41.474142 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:41.474142 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:41.474142 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:41.474142 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:41.474142 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:41.474142 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:41.475026 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:41.475026 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:41.475026 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:41.475026 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:41.475026 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:41.475026 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:41.475026 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:41.475026 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:41.475026 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:41.475026 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:41.475026 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:41.475026 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:41.475026 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:41.475026 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:41.475026 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:41.475026 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:41.475026 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:41.475922 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:41.475922 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:41.475922 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:41.475922 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:41.483305 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 423/846] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:41.484725 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f45619-e381-434c-8d4a-35f9c53532ce None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:41.491181 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 424/847] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 27 21:03:41 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:41.614815 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f45619-e381-434c-8d4a-35f9c53532ce None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:41.659832 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c4885-0e72-4527-b354-2f17a9b31186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:41.660506 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c4885-0e72-4527-b354-2f17a9b31186 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:41.660784 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c4885-0e72-4527-b354-2f17a9b31186 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:41.661090 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c4885-0e72-4527-b354-2f17a9b31186 None admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:41.661900 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32c4885-0e72-4527-b354-2f17a9b31186 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:41.662204 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32c4885-0e72-4527-b354-2f17a9b31186 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:41.663643 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32c4885-0e72-4527-b354-2f17a9b31186 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:41.678509 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 424/848] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:41.684197 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1bae3b-008b-4e9e-890b-ab75060574aa None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:41.715049 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a75035-39c7-4ccd-a1de-33012964708d None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:41.715581 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a75035-39c7-4ccd-a1de-33012964708d None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:41.715581 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a75035-39c7-4ccd-a1de-33012964708d None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:41.717334 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 425/849] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:41.722748 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bbe366-b419-40be-a09d-181460c27df9 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:41.723040 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bbe366-b419-40be-a09d-181460c27df9 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:41.723239 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bbe366-b419-40be-a09d-181460c27df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:41.742354 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-31bbe366-b419-40be-a09d-181460c27df9 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:41.743548 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-31bbe366-b419-40be-a09d-181460c27df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:41.782354 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:41.844969 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 426/850] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:03:41.847504 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1bae3b-008b-4e9e-890b-ab75060574aa None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:41.866100 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12485b7-8eef-4c1a-bd11-174df7a7f136 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:41.866395 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12485b7-8eef-4c1a-bd11-174df7a7f136 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:41.866625 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12485b7-8eef-4c1a-bd11-174df7a7f136 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:41.868268 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 427/851] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:03:41.872306 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12485b7-8eef-4c1a-bd11-174df7a7f136 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:41.894289 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce7549a-fec8-44f9-8131-54f475320fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:41.895091 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7549a-fec8-44f9-8131-54f475320fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:41.895374 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7549a-fec8-44f9-8131-54f475320fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:41.895649 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7549a-fec8-44f9-8131-54f475320fe6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:41.905066 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce7549a-fec8-44f9-8131-54f475320fe6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:41.906524 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce7549a-fec8-44f9-8131-54f475320fe6 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:41.929177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-bce7549a-fec8-44f9-8131-54f475320fe6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 27 21:03:41.929177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:41.929177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:41.929177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:41.929177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:41.929177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:41.929177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:41.929177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:41.929177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:41.929177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:41.929177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:41.929177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:41.929177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:41.929177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:41.929177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:41.929177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:41.930469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:41.930469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:41.930469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:41.930469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:41.930469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:41.930469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:41.930469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:41.930469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:41.930469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:41.930469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:41.930469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:41.930469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:41.930469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:41.930469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:41.930469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:41.930469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:41.930469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:41.930469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:41.931917 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:41.931917 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:41.931917 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:41.931917 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:41.931917 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:41.931917 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:41.931917 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:41.931917 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:41.931917 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 27 21:03:41.931917 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:41.931917 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 425/852] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 27 21:03:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:41.933786 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9debbcf0-102d-422c-9b21-133b7529ae4f None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:41.933786 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9debbcf0-102d-422c-9b21-133b7529ae4f None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:41.934188 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9debbcf0-102d-422c-9b21-133b7529ae4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:41.955526 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-9debbcf0-102d-422c-9b21-133b7529ae4f None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:41.956753 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-9debbcf0-102d-422c-9b21-133b7529ae4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:41.991706 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:42.006703 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12485b7-8eef-4c1a-bd11-174df7a7f136 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:42.041962 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 426/853] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:03:42.043205 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9debbcf0-102d-422c-9b21-133b7529ae4f None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:42.046808 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3873c471-26c5-4ea6-bbd0-417b4ca8ac27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:42.047559 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873c471-26c5-4ea6-bbd0-417b4ca8ac27 None admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:42.047903 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873c471-26c5-4ea6-bbd0-417b4ca8ac27 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:42.048254 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873c471-26c5-4ea6-bbd0-417b4ca8ac27 None admin] PATH_INFO: `/v3/services` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:42.049117 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3873c471-26c5-4ea6-bbd0-417b4ca8ac27 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:42.050935 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3873c471-26c5-4ea6-bbd0-417b4ca8ac27 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:42.068804 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 428/854] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Jun 27 21:03:41 2022] POST /identity/v3/services => generated 269 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:03:42.074323 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3873c471-26c5-4ea6-bbd0-417b4ca8ac27 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:42.115982 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:42.173687 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9debbcf0-102d-422c-9b21-133b7529ae4f None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:42.226680 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d869a-3d43-405a-8677-2a68bdb546a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:42.227279 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3873c471-26c5-4ea6-bbd0-417b4ca8ac27 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:42.227403 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d869a-3d43-405a-8677-2a68bdb546a9 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:42.227693 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d869a-3d43-405a-8677-2a68bdb546a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:42.227970 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d869a-3d43-405a-8677-2a68bdb546a9 None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:42.229040 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989d869a-3d43-405a-8677-2a68bdb546a9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:42.229542 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989d869a-3d43-405a-8677-2a68bdb546a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:42.231172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989d869a-3d43-405a-8677-2a68bdb546a9 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:42.255949 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 427/855] 10.222.0.13 () {56 vars in 1135 bytes} [Mon Jun 27 21:03:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:42.261794 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d869a-3d43-405a-8677-2a68bdb546a9 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:42.275419 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73bb1f3-3afe-4d12-93fb-18029246b75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:42.278300 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bb1f3-3afe-4d12-93fb-18029246b75d None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:42.278759 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bb1f3-3afe-4d12-93fb-18029246b75d None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:42.279189 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bb1f3-3afe-4d12-93fb-18029246b75d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:42.296895 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:42.300906 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73bb1f3-3afe-4d12-93fb-18029246b75d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:42.309191 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-c73bb1f3-3afe-4d12-93fb-18029246b75d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:42.309191 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:42.309191 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:42.309191 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:42.309191 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:42.309191 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:42.309191 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:42.309191 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:42.309191 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:42.309191 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:42.309191 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:42.309191 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:42.309191 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:42.309191 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:42.309191 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:42.309191 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:42.310235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:42.310235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:42.310235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:42.310235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:42.310235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:42.310235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:42.310235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:42.310235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:42.310235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:42.310235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:42.310235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:42.310235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:42.310235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:42.310235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:42.310235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:42.310235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:42.311284 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:42.311284 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:42.311284 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:42.311284 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:42.311284 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:42.311284 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:42.311284 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:42.311284 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:42.311284 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:42.311284 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:42.311284 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:03:42.311284 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:42.313686 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 429/856] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 27 21:03:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:42.319500 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73bb1f3-3afe-4d12-93fb-18029246b75d None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:42.402725 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d869a-3d43-405a-8677-2a68bdb546a9 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:42.441934 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:42.449711 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf41cd5-5e4d-48e6-b3e0-486a03e4a943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:42.450549 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf41cd5-5e4d-48e6-b3e0-486a03e4a943 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:42.450831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf41cd5-5e4d-48e6-b3e0-486a03e4a943 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:42.451105 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf41cd5-5e4d-48e6-b3e0-486a03e4a943 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:42.466498 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73bb1f3-3afe-4d12-93fb-18029246b75d None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:42.527372 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6269b327-c318-4006-92bb-79ef141918fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:42.528993 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6269b327-c318-4006-92bb-79ef141918fb None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:42.529451 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6269b327-c318-4006-92bb-79ef141918fb None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:42.529941 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6269b327-c318-4006-92bb-79ef141918fb None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:42.530868 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6269b327-c318-4006-92bb-79ef141918fb None admin] RBAC: Adding query filter params (name=service) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:42.531334 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6269b327-c318-4006-92bb-79ef141918fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:42.533552 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6269b327-c318-4006-92bb-79ef141918fb None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:42.546570 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 430/857] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 27 21:03:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:42.554517 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6269b327-c318-4006-92bb-79ef141918fb None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:42.555139 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf41cd5-5e4d-48e6-b3e0-486a03e4a943 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:42.556686 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf41cd5-5e4d-48e6-b3e0-486a03e4a943 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:42.585773 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:42.700541 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 428/858] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:42 2022] GET /identity/v3/auth/tokens => generated 1261 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 21:03:42.706424 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf41cd5-5e4d-48e6-b3e0-486a03e4a943 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:42.717538 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6269b327-c318-4006-92bb-79ef141918fb None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:42.768598 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e8f0d0-ded9-4f82-8495-71433428e3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:42.769435 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8f0d0-ded9-4f82-8495-71433428e3f7 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:42.769713 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8f0d0-ded9-4f82-8495-71433428e3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:42.769981 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8f0d0-ded9-4f82-8495-71433428e3f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:42.786531 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e8f0d0-ded9-4f82-8495-71433428e3f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:42.787887 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e8f0d0-ded9-4f82-8495-71433428e3f7 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:42.797913 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:42.813303 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-55e8f0d0-ded9-4f82-8495-71433428e3f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:42.813303 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:42.813303 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:42.813303 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:42.813303 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:42.813303 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:42.813303 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:42.813303 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:42.813303 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:42.813303 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:42.813303 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:42.813303 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:42.813303 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:42.813303 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:42.813303 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:42.814364 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:42.814364 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:42.814364 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:42.814364 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:42.814364 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:42.814364 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:42.814364 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:42.814364 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:42.814364 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:42.814364 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:42.814364 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:42.814364 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:42.814364 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:42.814364 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:42.814364 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:42.814364 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:42.814364 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:42.815349 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:42.815349 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:42.815349 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:42.815349 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:42.821158 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 431/859] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:42.827306 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e8f0d0-ded9-4f82-8495-71433428e3f7 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:42.857078 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf41cd5-5e4d-48e6-b3e0-486a03e4a943 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:42.888731 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:42.916872 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ad4e7f-e8da-4409-bc23-eb244ce6c0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:42.918513 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad4e7f-e8da-4409-bc23-eb244ce6c0b7 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:42.918898 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad4e7f-e8da-4409-bc23-eb244ce6c0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:42.919267 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad4e7f-e8da-4409-bc23-eb244ce6c0b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:42.934081 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ad4e7f-e8da-4409-bc23-eb244ce6c0b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:42.936965 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ad4e7f-e8da-4409-bc23-eb244ce6c0b7 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:42.957804 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-b8ad4e7f-e8da-4409-bc23-eb244ce6c0b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:42.957804 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:42.957804 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:42.957804 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:42.957804 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:42.957804 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:42.957804 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:42.957804 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:42.957804 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:42.957804 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:42.957804 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:42.957804 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:42.957804 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:42.957804 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:42.957804 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:42.959087 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:42.959087 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:42.959087 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:42.959087 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:42.959087 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:42.959087 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:42.959087 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:42.959087 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:42.959087 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:42.959087 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:42.959087 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:42.959087 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:42.959087 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:42.959087 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:42.959087 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:42.959087 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:42.959087 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:42.960685 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:42.960685 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:42.960685 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:42.960685 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:42.964786 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 429/860] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:42.967033 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ad4e7f-e8da-4409-bc23-eb244ce6c0b7 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:42.992428 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e8f0d0-ded9-4f82-8495-71433428e3f7 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:43.039974 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6649ac51-9d8b-45b1-92c9-775fa221b5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:43.041278 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649ac51-9d8b-45b1-92c9-775fa221b5c3 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:43.042970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649ac51-9d8b-45b1-92c9-775fa221b5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:43.043508 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649ac51-9d8b-45b1-92c9-775fa221b5c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:43.044468 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6649ac51-9d8b-45b1-92c9-775fa221b5c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:43.045113 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6649ac51-9d8b-45b1-92c9-775fa221b5c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:43.046712 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6649ac51-9d8b-45b1-92c9-775fa221b5c3 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:43.064341 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 432/861] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:43.070318 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6649ac51-9d8b-45b1-92c9-775fa221b5c3 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:43.106093 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ad4e7f-e8da-4409-bc23-eb244ce6c0b7 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:43.150486 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:43.153354 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9441a0a-c83c-4c25-b2fb-fc0300145f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:43.154049 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9441a0a-c83c-4c25-b2fb-fc0300145f24 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:43.154332 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9441a0a-c83c-4c25-b2fb-fc0300145f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:43.154615 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9441a0a-c83c-4c25-b2fb-fc0300145f24 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:43.164317 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9441a0a-c83c-4c25-b2fb-fc0300145f24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:43.164317 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9441a0a-c83c-4c25-b2fb-fc0300145f24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:43.164317 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9441a0a-c83c-4c25-b2fb-fc0300145f24 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:43.173444 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 430/862] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:43.181237 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9441a0a-c83c-4c25-b2fb-fc0300145f24 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:43.220987 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:43.232588 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6649ac51-9d8b-45b1-92c9-775fa221b5c3 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:43.271400 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3274cd-24f6-4da5-92ea-ca42e06b88cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:43.272616 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3274cd-24f6-4da5-92ea-ca42e06b88cc None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:43.273126 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3274cd-24f6-4da5-92ea-ca42e06b88cc None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:43.273618 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3274cd-24f6-4da5-92ea-ca42e06b88cc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:43.283475 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3274cd-24f6-4da5-92ea-ca42e06b88cc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:43.286260 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3274cd-24f6-4da5-92ea-ca42e06b88cc None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:43.299981 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9441a0a-c83c-4c25-b2fb-fc0300145f24 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:43.305704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-df3274cd-24f6-4da5-92ea-ca42e06b88cc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 21:03:43.305704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:43.305704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:43.305704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:43.305704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:43.305704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:43.305704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:43.305704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:43.305704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:43.305704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:43.305704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:43.305704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:43.305704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:43.305704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:43.305704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:43.305704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:43.307095 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:43.307095 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:43.307095 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:43.307095 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:43.307095 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:43.307095 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:43.307095 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:43.307095 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:43.307095 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:43.307095 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:43.307095 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:43.307095 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:43.307095 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:43.307095 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:43.307095 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:43.307095 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:43.307095 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:43.307095 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:43.310019 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:43.310019 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:43.310019 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:43.310019 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:43.310019 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:43.310019 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:43.310019 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:43.310019 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:43.310019 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 21:03:43.310019 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:43.317806 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 433/863] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Jun 27 21:03:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:43.319171 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3274cd-24f6-4da5-92ea-ca42e06b88cc None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:43.340876 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffbc507-6040-47c0-a11e-9486ceb13caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:43.341810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbc507-6040-47c0-a11e-9486ceb13caa None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:43.342098 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbc507-6040-47c0-a11e-9486ceb13caa None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:43.342381 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbc507-6040-47c0-a11e-9486ceb13caa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:43.350408 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffbc507-6040-47c0-a11e-9486ceb13caa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:43.352350 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffbc507-6040-47c0-a11e-9486ceb13caa None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:43.368046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-bffbc507-6040-47c0-a11e-9486ceb13caa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:43.368046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:43.368046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:43.368046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:43.368046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:43.368046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:43.368046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:43.368046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:43.368046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:43.368046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:43.368046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:43.368046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:43.368046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:43.368046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:43.368046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:43.369054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:43.369054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:43.369054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:43.369054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:43.369054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:43.369054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:43.369054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:43.369054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:43.369054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:43.369054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:43.369054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:43.369054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:43.369054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:43.369054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:43.369054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:43.369054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:43.369054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:43.370892 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:43.370892 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:43.370892 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:43.370892 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:43.370892 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:43.370892 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:43.370892 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:43.370892 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:43.374740 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 431/864] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:43.375760 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffbc507-6040-47c0-a11e-9486ceb13caa None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:43.439380 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3274cd-24f6-4da5-92ea-ca42e06b88cc None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:43.458406 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:43.477242 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aff6a4-7bb7-44d6-8942-df6715c55ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:43.478310 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aff6a4-7bb7-44d6-8942-df6715c55ee3 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:43.478795 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aff6a4-7bb7-44d6-8942-df6715c55ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:43.479353 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aff6a4-7bb7-44d6-8942-df6715c55ee3 None admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:43.480367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:43.480982 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aff6a4-7bb7-44d6-8942-df6715c55ee3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:43.481504 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aff6a4-7bb7-44d6-8942-df6715c55ee3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:43.483487 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aff6a4-7bb7-44d6-8942-df6715c55ee3 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:43.491287 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffbc507-6040-47c0-a11e-9486ceb13caa None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:43.509096 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 434/865] 10.222.0.13 () {56 vars in 1149 bytes} [Mon Jun 27 21:03:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:43.510482 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aff6a4-7bb7-44d6-8942-df6715c55ee3 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:43.530532 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6feb80-c817-401d-99f8-32343be7b1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:43.531207 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6feb80-c817-401d-99f8-32343be7b1d1 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:43.531488 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6feb80-c817-401d-99f8-32343be7b1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:43.531763 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6feb80-c817-401d-99f8-32343be7b1d1 None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:43.532674 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6feb80-c817-401d-99f8-32343be7b1d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:43.533024 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6feb80-c817-401d-99f8-32343be7b1d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:43.534600 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6feb80-c817-401d-99f8-32343be7b1d1 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:43.553114 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 432/866] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:43.554291 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6feb80-c817-401d-99f8-32343be7b1d1 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:43.626568 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aff6a4-7bb7-44d6-8942-df6715c55ee3 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:43.665622 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629e8adb-5ef2-4af4-8040-4655fc5fb63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:43.667021 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e8adb-5ef2-4af4-8040-4655fc5fb63d None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:43.668325 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e8adb-5ef2-4af4-8040-4655fc5fb63d None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:43.668325 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e8adb-5ef2-4af4-8040-4655fc5fb63d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:43.669734 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6feb80-c817-401d-99f8-32343be7b1d1 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:43.707436 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbc0091-76a1-44ed-9398-03bdd668779d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:43.708913 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc0091-76a1-44ed-9398-03bdd668779d None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:43.709441 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc0091-76a1-44ed-9398-03bdd668779d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:43.709866 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc0091-76a1-44ed-9398-03bdd668779d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:43.710863 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbc0091-76a1-44ed-9398-03bdd668779d None admin] RBAC: Adding query filter params (role.id=d336dd7b746c4488b33289c00302e4d5, scope.project.id=334438c929c44e489e9a3b93e6d37231, user.id=8bed6b231ee647d8a350da055d8e59d7) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:43.711346 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbc0091-76a1-44ed-9398-03bdd668779d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:43.712838 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbc0091-76a1-44ed-9398-03bdd668779d None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:43.718538 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:43.724999 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 433/867] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:03:43 2022] GET /identity/v3/role_assignments?user.id=8bed6b231ee647d8a350da055d8e59d7&scope.project.id=334438c929c44e489e9a3b93e6d37231&role.id=d336dd7b746c4488b33289c00302e4d5 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:43.724999 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb13ced-9cd3-420f-ba41-d997e792e338 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:43.724999 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb13ced-9cd3-420f-ba41-d997e792e338 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:43.724999 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb13ced-9cd3-420f-ba41-d997e792e338 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:43.732959 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 434/868] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:43.734925 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b3365-31c5-4df9-8da5-54398882002c None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:43.735214 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b3365-31c5-4df9-8da5-54398882002c None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:43.735492 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b3365-31c5-4df9-8da5-54398882002c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:43.757219 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629e8adb-5ef2-4af4-8040-4655fc5fb63d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:43.758570 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629e8adb-5ef2-4af4-8040-4655fc5fb63d None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:43.765595 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-372b3365-31c5-4df9-8da5-54398882002c None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:43.767151 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-372b3365-31c5-4df9-8da5-54398882002c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:43.782705 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:43.858345 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 435/869] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:43 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:03:43.879065 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372b3365-31c5-4df9-8da5-54398882002c None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:43.898916 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 435/870] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:43 2022] GET /identity/v3/auth/tokens => generated 1261 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 21:03:43.906610 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629e8adb-5ef2-4af4-8040-4655fc5fb63d None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:44.014022 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372b3365-31c5-4df9-8da5-54398882002c None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:44.019491 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:44.061639 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629e8adb-5ef2-4af4-8040-4655fc5fb63d None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:44.067695 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e421ba-16c8-4b48-878b-c44c8abcc677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:44.068824 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e421ba-16c8-4b48-878b-c44c8abcc677 None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:44.070782 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e421ba-16c8-4b48-878b-c44c8abcc677 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:44.071272 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e421ba-16c8-4b48-878b-c44c8abcc677 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:44.072708 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e421ba-16c8-4b48-878b-c44c8abcc677 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:44.075386 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e421ba-16c8-4b48-878b-c44c8abcc677 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:44.103653 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 436/871] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Jun 27 21:03:43 2022] POST /identity/v3/services => generated 251 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:03:44.119576 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e77c137-51ad-405b-9e93-e9efaf40968b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:44.121633 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e77c137-51ad-405b-9e93-e9efaf40968b None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:44.122149 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e77c137-51ad-405b-9e93-e9efaf40968b None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:44.122634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e77c137-51ad-405b-9e93-e9efaf40968b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:44.145311 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e77c137-51ad-405b-9e93-e9efaf40968b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:44.148234 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e77c137-51ad-405b-9e93-e9efaf40968b None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:44.185715 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-4e77c137-51ad-405b-9e93-e9efaf40968b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:44.185715 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:44.185715 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:44.185715 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:44.185715 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:44.185715 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:44.185715 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:44.185715 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:44.185715 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:44.185715 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:44.185715 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:44.185715 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:44.185715 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:44.185715 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:44.185715 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:44.186733 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:44.186733 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:44.186733 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:44.186733 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:44.186733 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:44.186733 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:44.186733 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:44.186733 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:44.186733 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:44.186733 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:44.186733 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:44.186733 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:44.186733 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:44.186733 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:44.186733 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:44.186733 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:44.186733 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:44.187606 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:44.187606 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:44.187606 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:44.187606 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:44.188968 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 436/872] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:44.199074 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e421ba-16c8-4b48-878b-c44c8abcc677 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:44.217359 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:44.367836 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d09a7-d1b4-4120-bc3e-41e3fa823547 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:44.368168 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d09a7-d1b4-4120-bc3e-41e3fa823547 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:44.368393 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d09a7-d1b4-4120-bc3e-41e3fa823547 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:44.370496 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 437/873] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:44.375984 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f36239-89da-4c04-9a7b-bf29b0fb0159 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:44.376233 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f36239-89da-4c04-9a7b-bf29b0fb0159 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:44.376468 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f36239-89da-4c04-9a7b-bf29b0fb0159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:44.394181 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e421ba-16c8-4b48-878b-c44c8abcc677 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:44.394539 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-c5f36239-89da-4c04-9a7b-bf29b0fb0159 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:44.395688 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-c5f36239-89da-4c04-9a7b-bf29b0fb0159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:44.447342 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b7bc12-d041-4443-87db-a949261b2257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:44.449000 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7bc12-d041-4443-87db-a949261b2257 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:44.449372 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7bc12-d041-4443-87db-a949261b2257 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:44.449714 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7bc12-d041-4443-87db-a949261b2257 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:44.450517 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b7bc12-d041-4443-87db-a949261b2257 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:44.450883 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b7bc12-d041-4443-87db-a949261b2257 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:44.452427 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b7bc12-d041-4443-87db-a949261b2257 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:44.469495 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 437/874] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:44.475819 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b7bc12-d041-4443-87db-a949261b2257 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:44.509522 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 438/875] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:44 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:03:44.532294 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f36239-89da-4c04-9a7b-bf29b0fb0159 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:44.571273 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:44.611558 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:44.612497 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b7bc12-d041-4443-87db-a949261b2257 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:44.656342 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a51bfc5-6474-4b2c-b5a2-1a3370294a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:44.657527 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51bfc5-6474-4b2c-b5a2-1a3370294a51 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:44.658039 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51bfc5-6474-4b2c-b5a2-1a3370294a51 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:44.658537 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51bfc5-6474-4b2c-b5a2-1a3370294a51 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:44.663543 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f36239-89da-4c04-9a7b-bf29b0fb0159 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:44.666716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a51bfc5-6474-4b2c-b5a2-1a3370294a51 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:44.668592 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a51bfc5-6474-4b2c-b5a2-1a3370294a51 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:44.684619 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-9a51bfc5-6474-4b2c-b5a2-1a3370294a51 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:44.684619 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:44.684619 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:44.684619 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:44.684619 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:44.684619 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:44.684619 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:44.684619 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:44.684619 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:44.684619 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:44.684619 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:44.684619 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:44.684619 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:44.684619 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:44.684619 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:44.685653 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:44.685653 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:44.685653 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:44.685653 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:44.685653 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:44.685653 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:44.685653 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:44.685653 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:44.685653 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:44.685653 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:44.685653 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:44.685653 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:44.685653 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:44.685653 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:44.685653 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:44.685653 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:44.685653 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:44.686632 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:44.686632 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:44.686632 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:44.686632 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:44.686632 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:44.686632 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:44.686632 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:44.686632 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:44.689205 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 438/876] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:44.699264 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a51bfc5-6474-4b2c-b5a2-1a3370294a51 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:44.703125 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d341963-8861-479b-af0b-32b7dda095c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:44.704152 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d341963-8861-479b-af0b-32b7dda095c6 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:44.704473 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d341963-8861-479b-af0b-32b7dda095c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:44.704754 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d341963-8861-479b-af0b-32b7dda095c6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:44.705621 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d341963-8861-479b-af0b-32b7dda095c6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:44.707531 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d341963-8861-479b-af0b-32b7dda095c6 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:44.721423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-4d341963-8861-479b-af0b-32b7dda095c6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 21:03:44.721423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:44.721423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:03:44.721423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:03:44.721423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:03:44.721423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:03:44.721423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:03:44.721423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:44.721423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 21:03:44.721423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:44.721423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:03:44.721423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:44.721423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:44.721423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:44.721423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:44.721423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:44.722938 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:44.722938 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:44.722938 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:44.722938 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:44.722938 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:44.722938 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:44.722938 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:44.722938 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:03:44.722938 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:03:44.722938 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:03:44.722938 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:03:44.722938 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:44.722938 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:44.722938 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:44.722938 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:44.722938 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:44.723824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:44.723824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:44.723824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:44.723824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:44.723824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:44.723824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:44.723824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:44.723824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:44.723824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:44.723824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:44.723824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:44.723824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:03:44.723824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:44.723824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 21:03:44.723824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:44.723824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 439/877] 10.222.0.13 () {56 vars in 1132 bytes} [Mon Jun 27 21:03:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:44.731790 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d341963-8861-479b-af0b-32b7dda095c6 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:44.867473 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d341963-8861-479b-af0b-32b7dda095c6 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:44.889886 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:44.910392 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3d1fd1-1065-4504-b5ab-1f17aef96be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:44.911492 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d1fd1-1065-4504-b5ab-1f17aef96be6 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:44.911989 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d1fd1-1065-4504-b5ab-1f17aef96be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:44.912516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d1fd1-1065-4504-b5ab-1f17aef96be6 None admin] PATH_INFO: `/v3/services` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:44.914208 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3d1fd1-1065-4504-b5ab-1f17aef96be6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:44.914770 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3d1fd1-1065-4504-b5ab-1f17aef96be6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:44.916906 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3d1fd1-1065-4504-b5ab-1f17aef96be6 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:44.928681 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 440/878] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Jun 27 21:03:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:44.930766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3d1fd1-1065-4504-b5ab-1f17aef96be6 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:44.953256 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a51bfc5-6474-4b2c-b5a2-1a3370294a51 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:44.985750 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:44.995680 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df34f10-0113-44c8-83b5-e72dd2f37fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:44.996860 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df34f10-0113-44c8-83b5-e72dd2f37fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:44.997440 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df34f10-0113-44c8-83b5-e72dd2f37fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:44.997939 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df34f10-0113-44c8-83b5-e72dd2f37fd9 None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:44.998954 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df34f10-0113-44c8-83b5-e72dd2f37fd9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:44.999501 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df34f10-0113-44c8-83b5-e72dd2f37fd9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:45.001213 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df34f10-0113-44c8-83b5-e72dd2f37fd9 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:45.015660 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 439/879] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:45.021699 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df34f10-0113-44c8-83b5-e72dd2f37fd9 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:45.062994 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3d1fd1-1065-4504-b5ab-1f17aef96be6 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:45.107600 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab144e7-697b-4430-ba73-90857a3f73c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:45.108636 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab144e7-697b-4430-ba73-90857a3f73c9 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:45.109223 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab144e7-697b-4430-ba73-90857a3f73c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:45.109717 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab144e7-697b-4430-ba73-90857a3f73c9 None admin] PATH_INFO: `/v3/services` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:45.110635 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab144e7-697b-4430-ba73-90857a3f73c9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:45.111136 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab144e7-697b-4430-ba73-90857a3f73c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:45.113257 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab144e7-697b-4430-ba73-90857a3f73c9 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:45.125174 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 441/880] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Jun 27 21:03:44 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:45.127031 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab144e7-697b-4430-ba73-90857a3f73c9 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:45.149021 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df34f10-0113-44c8-83b5-e72dd2f37fd9 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:45.187030 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:45.190144 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dee751a-b8d9-4473-89c1-f30047027135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:45.191210 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee751a-b8d9-4473-89c1-f30047027135 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:45.191720 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee751a-b8d9-4473-89c1-f30047027135 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:45.192193 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee751a-b8d9-4473-89c1-f30047027135 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:45.193300 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dee751a-b8d9-4473-89c1-f30047027135 None admin] RBAC: Adding query filter params (role.id=709aef6b85524eb2a8e9986a2bd9525b, scope.project.id=334438c929c44e489e9a3b93e6d37231, user.id=f67110d591a94537a3df6a2d835370f9) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:45.193818 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dee751a-b8d9-4473-89c1-f30047027135 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:45.195580 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dee751a-b8d9-4473-89c1-f30047027135 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:45.209266 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 440/881] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:03:45 2022] GET /identity/v3/role_assignments?user.id=f67110d591a94537a3df6a2d835370f9&scope.project.id=334438c929c44e489e9a3b93e6d37231&role.id=709aef6b85524eb2a8e9986a2bd9525b => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:45.250464 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab144e7-697b-4430-ba73-90857a3f73c9 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:45.289597 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a0ee4-c7b8-4153-8015-cf6a003b0dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:45.290283 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a0ee4-c7b8-4153-8015-cf6a003b0dbc None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:45.290570 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a0ee4-c7b8-4153-8015-cf6a003b0dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:45.290845 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a0ee4-c7b8-4153-8015-cf6a003b0dbc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:45.291734 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7a0ee4-c7b8-4153-8015-cf6a003b0dbc None admin] RBAC: Adding query filter params (interface=public, service_id=4a3efef96c9241b3a4b67b37110713e8, region_id=RegionOne) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:45.292050 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7a0ee4-c7b8-4153-8015-cf6a003b0dbc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:45.293650 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7a0ee4-c7b8-4153-8015-cf6a003b0dbc None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:45.300286 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 442/882] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Jun 27 21:03:45 2022] GET /identity/v3/endpoints?service_id=4a3efef96c9241b3a4b67b37110713e8&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:45.306094 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dee751a-b8d9-4473-89c1-f30047027135 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:45.374442 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:45.424335 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dee751a-b8d9-4473-89c1-f30047027135 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:45.467189 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9809203-4e32-4133-81b6-6c5d7b48d0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:45.468208 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9809203-4e32-4133-81b6-6c5d7b48d0f7 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:45.468687 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9809203-4e32-4133-81b6-6c5d7b48d0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:45.469212 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9809203-4e32-4133-81b6-6c5d7b48d0f7 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:45.470140 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9809203-4e32-4133-81b6-6c5d7b48d0f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:45.471532 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9809203-4e32-4133-81b6-6c5d7b48d0f7 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:45.478722 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 441/883] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Jun 27 21:03:45 2022] GET /identity/v3/services => generated 1268 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:03:45.972614 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfbd287-eb9d-4467-9d51-89894ac46418 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:45.973221 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfbd287-eb9d-4467-9d51-89894ac46418 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:45.974301 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfbd287-eb9d-4467-9d51-89894ac46418 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:45.975896 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 443/884] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:45.983631 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1529f-9342-477c-a28c-c38832b59f72 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:45.984981 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1529f-9342-477c-a28c-c38832b59f72 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:45.985722 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1529f-9342-477c-a28c-c38832b59f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:46.005367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-60f1529f-9342-477c-a28c-c38832b59f72 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:46.006934 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-60f1529f-9342-477c-a28c-c38832b59f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:46.089269 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 442/885] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:45 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:03:46.127679 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfbd287-eb9d-4467-9d51-89894ac46418 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:46.312019 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfbd287-eb9d-4467-9d51-89894ac46418 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:46.332531 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:46.361512 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16524e-0c7d-457d-8579-7e6aa29fe6b0 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:46.361833 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16524e-0c7d-457d-8579-7e6aa29fe6b0 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:46.362156 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16524e-0c7d-457d-8579-7e6aa29fe6b0 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:46.367434 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 443/886] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:46.369050 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae25a90-43db-4999-95bf-fc03a79537f9 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:46.369334 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae25a90-43db-4999-95bf-fc03a79537f9 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:46.370057 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae25a90-43db-4999-95bf-fc03a79537f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:46.374558 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cfbc17-f698-451c-9b63-9f9f153b2a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:46.376298 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cfbc17-f698-451c-9b63-9f9f153b2a2f None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:46.376770 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cfbc17-f698-451c-9b63-9f9f153b2a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:46.377283 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cfbc17-f698-451c-9b63-9f9f153b2a2f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:46.378512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cfbc17-f698-451c-9b63-9f9f153b2a2f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:46.381080 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cfbc17-f698-451c-9b63-9f9f153b2a2f None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:46.387489 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-9ae25a90-43db-4999-95bf-fc03a79537f9 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:46.388836 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-9ae25a90-43db-4999-95bf-fc03a79537f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:46.407048 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-47cfbc17-f698-451c-9b63-9f9f153b2a2f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 21:03:46.407048 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:46.407048 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:03:46.407048 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:03:46.407048 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:03:46.407048 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:03:46.407048 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:03:46.407048 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:46.407048 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 21:03:46.407048 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:46.407048 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:03:46.407048 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:46.407048 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:46.407048 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:46.407048 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:46.407048 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:46.407048 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:46.409327 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:46.409327 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:46.409327 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:46.409327 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:46.409327 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:46.409327 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:46.409327 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:03:46.409327 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:03:46.409327 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:03:46.409327 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:03:46.409327 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:46.409327 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:46.409327 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:46.409327 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:46.409327 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:46.409327 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:46.411107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:46.411107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:46.411107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:46.411107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:46.411107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:46.411107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:46.411107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:46.411107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:46.411107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:46.411107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:46.411107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:03:46.411107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:46.411107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 21:03:46.411107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:46.411107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 444/887] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Jun 27 21:03:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:46.421597 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cfbc17-f698-451c-9b63-9f9f153b2a2f None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:46.437773 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:46.468975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 444/888] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:46 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:03:46.491953 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae25a90-43db-4999-95bf-fc03a79537f9 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:46.538978 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cfbc17-f698-451c-9b63-9f9f153b2a2f None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:46.580767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4393daa7-285f-41e9-90f9-b5d1a3e0430c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:46.581436 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393daa7-285f-41e9-90f9-b5d1a3e0430c None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:46.581717 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393daa7-285f-41e9-90f9-b5d1a3e0430c None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:46.582044 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393daa7-285f-41e9-90f9-b5d1a3e0430c None admin] PATH_INFO: `/v3/services` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:46.582875 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4393daa7-285f-41e9-90f9-b5d1a3e0430c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:46.583244 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4393daa7-285f-41e9-90f9-b5d1a3e0430c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:46.584704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4393daa7-285f-41e9-90f9-b5d1a3e0430c None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:46.595500 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 445/889] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Jun 27 21:03:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:46.599282 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4393daa7-285f-41e9-90f9-b5d1a3e0430c None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:46.611100 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae25a90-43db-4999-95bf-fc03a79537f9 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:46.652354 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:46.657400 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e58baf1-5e3e-4db1-99a0-6e23b7749f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:46.659031 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58baf1-5e3e-4db1-99a0-6e23b7749f38 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:46.659670 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58baf1-5e3e-4db1-99a0-6e23b7749f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:46.660220 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58baf1-5e3e-4db1-99a0-6e23b7749f38 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:46.661150 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e58baf1-5e3e-4db1-99a0-6e23b7749f38 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:46.663570 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e58baf1-5e3e-4db1-99a0-6e23b7749f38 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:46.683389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-8e58baf1-5e3e-4db1-99a0-6e23b7749f38 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 21:03:46.683389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:46.683389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:03:46.683389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:03:46.683389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:03:46.683389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:03:46.683389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:03:46.683389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:46.683389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 21:03:46.683389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:46.683389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:03:46.683389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:46.683389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:46.683389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:46.683389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:46.683389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:46.683389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:46.684682 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:46.684682 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:46.684682 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:46.684682 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:46.684682 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:46.684682 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:46.684682 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:03:46.684682 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:03:46.684682 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:03:46.684682 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:03:46.684682 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:46.684682 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:46.684682 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:46.684682 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:46.684682 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:46.684682 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:46.685729 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:46.685729 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:46.685729 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:46.685729 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:46.685729 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:46.685729 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:46.685729 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:46.685729 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:46.685729 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:46.685729 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:46.685729 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:03:46.685729 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:46.685729 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 21:03:46.685729 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:46.687918 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 445/890] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 27 21:03:46 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:46.694286 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e58baf1-5e3e-4db1-99a0-6e23b7749f38 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:46.744743 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:46.747038 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4393daa7-285f-41e9-90f9-b5d1a3e0430c None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:46.785491 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eaa24d-ac64-408f-ae66-8de4f54d2ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:46.786471 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eaa24d-ac64-408f-ae66-8de4f54d2ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:46.786886 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eaa24d-ac64-408f-ae66-8de4f54d2ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:46.787342 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eaa24d-ac64-408f-ae66-8de4f54d2ae8 None admin] PATH_INFO: `/v3/services` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:46.788225 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eaa24d-ac64-408f-ae66-8de4f54d2ae8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:46.788692 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eaa24d-ac64-408f-ae66-8de4f54d2ae8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:46.790277 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eaa24d-ac64-408f-ae66-8de4f54d2ae8 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:46.796540 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 446/891] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Jun 27 21:03:46 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:46.813619 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e58baf1-5e3e-4db1-99a0-6e23b7749f38 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:46.855011 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af1605-3155-471a-9eaf-7c807b25c2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:46.855648 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af1605-3155-471a-9eaf-7c807b25c2e2 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:46.855984 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af1605-3155-471a-9eaf-7c807b25c2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:46.856266 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af1605-3155-471a-9eaf-7c807b25c2e2 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:46.857057 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46af1605-3155-471a-9eaf-7c807b25c2e2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:46.857371 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46af1605-3155-471a-9eaf-7c807b25c2e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:46.858827 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46af1605-3155-471a-9eaf-7c807b25c2e2 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:46.865135 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 446/892] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 27 21:03:46 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:46.871199 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eaa24d-ac64-408f-ae66-8de4f54d2ae8 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:46.997622 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eaa24d-ac64-408f-ae66-8de4f54d2ae8 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:47.000310 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:47.045597 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd37733-4d8d-4fed-99b1-7de2be9ada1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:47.046680 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd37733-4d8d-4fed-99b1-7de2be9ada1e None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:47.047094 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd37733-4d8d-4fed-99b1-7de2be9ada1e None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:47.047419 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd37733-4d8d-4fed-99b1-7de2be9ada1e None admin] PATH_INFO: `/v3/services` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:47.048152 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd37733-4d8d-4fed-99b1-7de2be9ada1e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:47.048593 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd37733-4d8d-4fed-99b1-7de2be9ada1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:47.050287 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd37733-4d8d-4fed-99b1-7de2be9ada1e None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:47.057885 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 447/893] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 27 21:03:46 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:47.063410 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af1605-3155-471a-9eaf-7c807b25c2e2 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:47.183938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af1605-3155-471a-9eaf-7c807b25c2e2 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:47.185582 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:47.221561 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e73ad1b-e7e6-4647-ae63-96577897477c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:47.222270 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73ad1b-e7e6-4647-ae63-96577897477c None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:47.222553 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73ad1b-e7e6-4647-ae63-96577897477c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:47.222827 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73ad1b-e7e6-4647-ae63-96577897477c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:47.223629 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e73ad1b-e7e6-4647-ae63-96577897477c None admin] RBAC: Adding query filter params (interface=public, service_id=cd926f00a0f049dfbebb991539fec518, region_id=RegionOne) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:47.223939 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e73ad1b-e7e6-4647-ae63-96577897477c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:47.225476 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e73ad1b-e7e6-4647-ae63-96577897477c None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:47.234554 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 447/894] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Jun 27 21:03:47 2022] GET /identity/v3/endpoints?service_id=cd926f00a0f049dfbebb991539fec518&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:47.240088 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd37733-4d8d-4fed-99b1-7de2be9ada1e None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:47.354348 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd37733-4d8d-4fed-99b1-7de2be9ada1e None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:47.391720 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66f055d-c0dd-4fe2-b7e1-fe54ce6e4138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:47.392402 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f055d-c0dd-4fe2-b7e1-fe54ce6e4138 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:47.392684 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f055d-c0dd-4fe2-b7e1-fe54ce6e4138 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:47.392984 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f055d-c0dd-4fe2-b7e1-fe54ce6e4138 None admin] PATH_INFO: `/v3/services` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:47.393684 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66f055d-c0dd-4fe2-b7e1-fe54ce6e4138 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:47.395061 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66f055d-c0dd-4fe2-b7e1-fe54ce6e4138 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:47.401626 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 448/895] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Jun 27 21:03:47 2022] GET /identity/v3/services => generated 1268 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:03:47.622271 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33571a22-d65c-4ced-9b6c-4b4530d636d6 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:47.622454 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33571a22-d65c-4ced-9b6c-4b4530d636d6 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:47.622740 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33571a22-d65c-4ced-9b6c-4b4530d636d6 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:47.624436 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 448/896] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:47.630043 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c5bca-c686-4c41-94ff-ed96c0fd0348 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:47.630334 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c5bca-c686-4c41-94ff-ed96c0fd0348 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:47.630601 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c5bca-c686-4c41-94ff-ed96c0fd0348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:47.650617 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-b97c5bca-c686-4c41-94ff-ed96c0fd0348 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:47.651896 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-b97c5bca-c686-4c41-94ff-ed96c0fd0348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:47.678614 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:47.687589 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a9b5a-bc89-400e-8adf-edbfd245717a None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:47.687935 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a9b5a-bc89-400e-8adf-edbfd245717a None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:47.688111 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a9b5a-bc89-400e-8adf-edbfd245717a None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:47.690086 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 449/897] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:47.695780 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc7425-2e9f-4b34-8859-78d8dcd1565b None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:47.696390 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc7425-2e9f-4b34-8859-78d8dcd1565b None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:47.696860 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc7425-2e9f-4b34-8859-78d8dcd1565b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:47.715325 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-59cc7425-2e9f-4b34-8859-78d8dcd1565b None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:47.716878 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-59cc7425-2e9f-4b34-8859-78d8dcd1565b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:47.741695 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 449/898] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:47 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:03:47.770933 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c5bca-c686-4c41-94ff-ed96c0fd0348 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:47.806567 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 450/899] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:47 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:03:47.830557 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc7425-2e9f-4b34-8859-78d8dcd1565b None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:47.902230 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c5bca-c686-4c41-94ff-ed96c0fd0348 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:47.945049 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012da111-74ed-48d1-ac7c-63459b73de4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:47.946291 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012da111-74ed-48d1-ac7c-63459b73de4f None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:47.946767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012da111-74ed-48d1-ac7c-63459b73de4f None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:47.947240 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012da111-74ed-48d1-ac7c-63459b73de4f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:47.955251 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:47.961276 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012da111-74ed-48d1-ac7c-63459b73de4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:47.961493 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc7425-2e9f-4b34-8859-78d8dcd1565b None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:47.971229 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-012da111-74ed-48d1-ac7c-63459b73de4f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 21:03:47.971229 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:47.971229 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:47.971229 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:47.971229 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:47.971229 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:47.971229 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:47.971229 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:47.971229 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:47.971229 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:47.971229 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:47.971229 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:47.971229 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:47.971229 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:47.971229 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:47.971229 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:47.972354 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:47.972354 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:47.972354 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:47.972354 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:47.972354 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:47.972354 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:47.972354 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:47.972354 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:47.972354 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:47.972354 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:47.972354 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:47.972354 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:47.972354 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:47.972354 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:47.972354 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:47.972354 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:47.973697 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:47.973697 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:47.973697 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:47.973697 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:47.973697 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:47.973697 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:47.973697 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:47.973697 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:47.973697 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:47.973697 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:47.973697 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 21:03:47.973697 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:47.981443 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 450/900] 10.222.0.13 () {56 vars in 1120 bytes} [Mon Jun 27 21:03:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:47.984026 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012da111-74ed-48d1-ac7c-63459b73de4f None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:48.002732 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572a3bf2-f56c-49c8-bae4-e9cb81bc253a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:48.003596 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a3bf2-f56c-49c8-bae4-e9cb81bc253a None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:48.003975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a3bf2-f56c-49c8-bae4-e9cb81bc253a None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:48.004256 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a3bf2-f56c-49c8-bae4-e9cb81bc253a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:48.004964 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572a3bf2-f56c-49c8-bae4-e9cb81bc253a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:48.006422 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572a3bf2-f56c-49c8-bae4-e9cb81bc253a None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:48.014343 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:48.030195 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-572a3bf2-f56c-49c8-bae4-e9cb81bc253a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 21:03:48.030195 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:48.030195 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:03:48.030195 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:03:48.030195 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:03:48.030195 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:03:48.030195 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:03:48.030195 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:48.030195 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 21:03:48.030195 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:48.030195 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:03:48.030195 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:48.030195 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:48.030195 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:48.030195 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:48.030195 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:48.031306 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:48.031306 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:48.031306 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:48.031306 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:48.031306 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:48.031306 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:48.031306 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:48.031306 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:03:48.031306 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:03:48.031306 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:03:48.031306 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:03:48.031306 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:48.031306 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:48.031306 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:48.031306 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:48.031306 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:48.032137 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:48.032137 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:48.032137 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:48.032137 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:48.032137 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:48.032137 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:48.032137 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:48.032137 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:48.032137 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:48.032137 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:48.032137 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:48.032137 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:03:48.032137 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:48.032137 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 27 21:03:48.032137 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:48.032137 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 451/901] 10.222.0.13 () {56 vars in 1132 bytes} [Mon Jun 27 21:03:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:48.039701 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572a3bf2-f56c-49c8-bae4-e9cb81bc253a None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:48.113554 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012da111-74ed-48d1-ac7c-63459b73de4f None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:48.158392 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f344042e-e942-4c8d-b062-a220fb567a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:48.159096 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344042e-e942-4c8d-b062-a220fb567a80 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:48.159376 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344042e-e942-4c8d-b062-a220fb567a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:48.159648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344042e-e942-4c8d-b062-a220fb567a80 None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:48.160708 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f344042e-e942-4c8d-b062-a220fb567a80 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:48.160881 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f344042e-e942-4c8d-b062-a220fb567a80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:48.162345 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f344042e-e942-4c8d-b062-a220fb567a80 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:48.176352 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 451/902] 10.222.0.13 () {56 vars in 1115 bytes} [Mon Jun 27 21:03:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:48.177753 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f344042e-e942-4c8d-b062-a220fb567a80 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:48.181179 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572a3bf2-f56c-49c8-bae4-e9cb81bc253a None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:48.220319 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a47c2-932b-4953-931e-b574d247644b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:48.221413 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a47c2-932b-4953-931e-b574d247644b None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:48.221890 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a47c2-932b-4953-931e-b574d247644b None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:48.222459 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a47c2-932b-4953-931e-b574d247644b None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:48.223371 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621a47c2-932b-4953-931e-b574d247644b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:48.223995 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621a47c2-932b-4953-931e-b574d247644b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:48.225833 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621a47c2-932b-4953-931e-b574d247644b None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:48.237104 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 452/903] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Jun 27 21:03:48 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:48.238390 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a47c2-932b-4953-931e-b574d247644b None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:48.266267 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:48.270185 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:48.291544 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f344042e-e942-4c8d-b062-a220fb567a80 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:48.328771 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86cc9f-5cd9-4ab4-a61b-021a32e8d052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:48.329621 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86cc9f-5cd9-4ab4-a61b-021a32e8d052 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:48.329904 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86cc9f-5cd9-4ab4-a61b-021a32e8d052 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:48.330179 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86cc9f-5cd9-4ab4-a61b-021a32e8d052 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:48.342635 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf86cc9f-5cd9-4ab4-a61b-021a32e8d052 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:48.344059 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf86cc9f-5cd9-4ab4-a61b-021a32e8d052 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:48.352538 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a47c2-932b-4953-931e-b574d247644b None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:48.363366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-cf86cc9f-5cd9-4ab4-a61b-021a32e8d052 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:48.363366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:48.363366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:48.363366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:48.363366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:48.363366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:48.363366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:48.363366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:48.363366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:48.363366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:48.363366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:48.363366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:48.363366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:48.363366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:48.363366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:48.364332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:48.364332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:48.364332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:48.364332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:48.364332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:48.364332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:48.364332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:48.364332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:48.364332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:48.364332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:48.364332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:48.364332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:48.364332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:48.364332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:48.364332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:48.364332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:48.364332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:48.370510 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:48.370510 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:48.370510 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:48.370510 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:48.370510 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 452/904] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:48.371032 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86cc9f-5cd9-4ab4-a61b-021a32e8d052 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:48.406912 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f2e2a-3b67-4fe8-bc70-1d45dd741ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:48.407987 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f2e2a-3b67-4fe8-bc70-1d45dd741ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:48.408668 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f2e2a-3b67-4fe8-bc70-1d45dd741ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:48.409283 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f2e2a-3b67-4fe8-bc70-1d45dd741ca6 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:48.410196 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2f2e2a-3b67-4fe8-bc70-1d45dd741ca6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:48.410727 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2f2e2a-3b67-4fe8-bc70-1d45dd741ca6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:48.412322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2f2e2a-3b67-4fe8-bc70-1d45dd741ca6 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:48.419646 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 453/905] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Jun 27 21:03:48 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:48.430646 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f2e2a-3b67-4fe8-bc70-1d45dd741ca6 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:48.490942 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86cc9f-5cd9-4ab4-a61b-021a32e8d052 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:48.529607 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73efa0cc-c40a-48b2-b443-dad68ba1693a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:48.530320 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73efa0cc-c40a-48b2-b443-dad68ba1693a None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:48.530603 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73efa0cc-c40a-48b2-b443-dad68ba1693a None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:48.530884 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73efa0cc-c40a-48b2-b443-dad68ba1693a None admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:48.531630 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73efa0cc-c40a-48b2-b443-dad68ba1693a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:48.531957 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73efa0cc-c40a-48b2-b443-dad68ba1693a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:48.533521 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73efa0cc-c40a-48b2-b443-dad68ba1693a None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:48.535465 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:48.550456 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 453/906] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:48.556132 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73efa0cc-c40a-48b2-b443-dad68ba1693a None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:48.559369 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:48.577190 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f2e2a-3b67-4fe8-bc70-1d45dd741ca6 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:48.628272 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bfe972-971a-4e46-881b-c5a188e5792f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:48.629063 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfe972-971a-4e46-881b-c5a188e5792f None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:48.629406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfe972-971a-4e46-881b-c5a188e5792f None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:48.630110 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfe972-971a-4e46-881b-c5a188e5792f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:48.631472 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bfe972-971a-4e46-881b-c5a188e5792f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:48.633578 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bfe972-971a-4e46-881b-c5a188e5792f None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:48.665042 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 454/907] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Jun 27 21:03:48 2022] POST /identity/v3/endpoints => generated 347 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:03:48.695098 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73efa0cc-c40a-48b2-b443-dad68ba1693a None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:48.743237 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee2c15-44e0-48fa-9553-2306a7f40297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:48.744104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee2c15-44e0-48fa-9553-2306a7f40297 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:48.744414 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee2c15-44e0-48fa-9553-2306a7f40297 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:48.744911 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee2c15-44e0-48fa-9553-2306a7f40297 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:48.753574 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aee2c15-44e0-48fa-9553-2306a7f40297 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:48.755061 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aee2c15-44e0-48fa-9553-2306a7f40297 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:48.775218 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-2aee2c15-44e0-48fa-9553-2306a7f40297 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 21:03:48.775218 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:48.775218 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:48.775218 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:48.775218 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:48.775218 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:48.775218 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:48.775218 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:48.775218 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:48.775218 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:48.775218 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:48.775218 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:48.775218 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:48.775218 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:48.775218 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:48.775218 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:48.776375 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:48.776375 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:48.776375 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:48.776375 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:48.776375 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:48.776375 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:48.776375 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:48.776375 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:48.776375 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:48.776375 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:48.776375 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:48.776375 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:48.776375 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:48.776375 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:48.776375 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:48.776375 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:48.776375 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:48.776375 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:48.777381 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:48.777381 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:48.777381 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:48.777381 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:48.777381 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:48.777381 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:48.777381 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:48.777381 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:48.777381 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 21:03:48.777381 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:48.777381 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 454/908] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Jun 27 21:03:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:48.786285 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bfe972-971a-4e46-881b-c5a188e5792f None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:48.916347 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bfe972-971a-4e46-881b-c5a188e5792f None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:48.953063 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3933e9-c06f-4129-a6d5-7f6bcfb46359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:48.953737 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3933e9-c06f-4129-a6d5-7f6bcfb46359 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:48.954017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3933e9-c06f-4129-a6d5-7f6bcfb46359 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:48.954291 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3933e9-c06f-4129-a6d5-7f6bcfb46359 None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:48.955141 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3933e9-c06f-4129-a6d5-7f6bcfb46359 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:48.955449 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3933e9-c06f-4129-a6d5-7f6bcfb46359 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:48.956911 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3933e9-c06f-4129-a6d5-7f6bcfb46359 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:48.977292 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 455/909] 10.222.0.13 () {56 vars in 1149 bytes} [Mon Jun 27 21:03:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:48.983305 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee2c15-44e0-48fa-9553-2306a7f40297 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:49.022084 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da0468-b97f-46ec-8910-e02ad0e8af4d None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:49.022374 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da0468-b97f-46ec-8910-e02ad0e8af4d None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:49.022639 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da0468-b97f-46ec-8910-e02ad0e8af4d None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:49.032136 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 456/910] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:49.034013 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278454b-440d-48eb-8c1c-742726ff9ed8 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:49.034299 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278454b-440d-48eb-8c1c-742726ff9ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:49.034577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278454b-440d-48eb-8c1c-742726ff9ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:49.040657 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:49.041763 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:49.053157 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-6278454b-440d-48eb-8c1c-742726ff9ed8 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:49.054908 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-6278454b-440d-48eb-8c1c-742726ff9ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:49.106893 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee2c15-44e0-48fa-9553-2306a7f40297 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:49.155195 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed62481-3e32-4058-b84d-64286c15931e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:49.155928 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed62481-3e32-4058-b84d-64286c15931e None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:49.156209 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed62481-3e32-4058-b84d-64286c15931e None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:49.156519 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed62481-3e32-4058-b84d-64286c15931e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:49.165177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 457/911] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:49 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:03:49.187299 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6278454b-440d-48eb-8c1c-742726ff9ed8 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:49.244038 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed62481-3e32-4058-b84d-64286c15931e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:49.246024 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed62481-3e32-4058-b84d-64286c15931e None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:49.296841 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6278454b-440d-48eb-8c1c-742726ff9ed8 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:49.301864 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:49.331199 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:49.336776 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843731ae-b27b-4a1e-9257-e36cf12f29f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:49.337509 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843731ae-b27b-4a1e-9257-e36cf12f29f6 None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:49.337796 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843731ae-b27b-4a1e-9257-e36cf12f29f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:49.338238 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843731ae-b27b-4a1e-9257-e36cf12f29f6 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:49.339056 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843731ae-b27b-4a1e-9257-e36cf12f29f6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:49.340604 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843731ae-b27b-4a1e-9257-e36cf12f29f6 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:49.356513 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 458/912] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Jun 27 21:03:49 2022] POST /identity/v3/services => generated 241 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:03:49.365910 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 455/913] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:48 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 27 21:03:49.371974 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843731ae-b27b-4a1e-9257-e36cf12f29f6 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:49.508418 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843731ae-b27b-4a1e-9257-e36cf12f29f6 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:49.548441 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f7c941-e164-4f0c-a81e-53802f698065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:49.549275 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7c941-e164-4f0c-a81e-53802f698065 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:49.549548 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7c941-e164-4f0c-a81e-53802f698065 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:49.549824 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7c941-e164-4f0c-a81e-53802f698065 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:49.563487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f7c941-e164-4f0c-a81e-53802f698065 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:49.565441 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f7c941-e164-4f0c-a81e-53802f698065 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:49.579507 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68775580-c48c-4ef8-a886-c47ac7d1af37 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:49.579826 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68775580-c48c-4ef8-a886-c47ac7d1af37 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:49.580087 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68775580-c48c-4ef8-a886-c47ac7d1af37 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:49.581758 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 456/914] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:03:49.584454 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-b2f7c941-e164-4f0c-a81e-53802f698065 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:49.584454 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:49.584454 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:49.584454 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:49.584454 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:49.584454 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:49.584454 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:49.584454 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:49.584454 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:49.584454 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:49.584454 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:49.584454 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:49.584454 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:49.584454 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:49.584454 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:49.585435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:49.585435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:49.585435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:49.585435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:49.585435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:49.585435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:49.585435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:49.585435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:49.585435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:49.585435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:49.585435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:49.585435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:49.585435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:49.585435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:49.585435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:49.585435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:49.585435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:49.586326 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:49.586326 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:49.586326 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:49.586326 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:49.586326 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 459/915] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:49.587221 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13939ceb-af54-489a-9774-164f5e19b276 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:49.587512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13939ceb-af54-489a-9774-164f5e19b276 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:49.587788 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13939ceb-af54-489a-9774-164f5e19b276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:49.593435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f7c941-e164-4f0c-a81e-53802f698065 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:49.610619 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-13939ceb-af54-489a-9774-164f5e19b276 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:49.611984 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-13939ceb-af54-489a-9774-164f5e19b276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:49.646109 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:49.700479 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 457/916] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:03:49.715643 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f7c941-e164-4f0c-a81e-53802f698065 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:49.722948 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13939ceb-af54-489a-9774-164f5e19b276 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:49.753578 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81018810-f6d1-4605-98b9-0adc6417848a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:49.754244 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81018810-f6d1-4605-98b9-0adc6417848a None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:49.754518 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81018810-f6d1-4605-98b9-0adc6417848a None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:49.754793 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81018810-f6d1-4605-98b9-0adc6417848a None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:49.755514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81018810-f6d1-4605-98b9-0adc6417848a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:49.755817 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81018810-f6d1-4605-98b9-0adc6417848a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:49.757220 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81018810-f6d1-4605-98b9-0adc6417848a None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:49.770906 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 460/917] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:49.778047 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81018810-f6d1-4605-98b9-0adc6417848a None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:49.822756 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:49.849614 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13939ceb-af54-489a-9774-164f5e19b276 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:49.896420 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cb820c-217a-400d-9916-4dff76fd24fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:49.897604 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb820c-217a-400d-9916-4dff76fd24fd None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:49.898732 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb820c-217a-400d-9916-4dff76fd24fd None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:49.899205 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb820c-217a-400d-9916-4dff76fd24fd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:49.900056 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cb820c-217a-400d-9916-4dff76fd24fd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:49.902141 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cb820c-217a-400d-9916-4dff76fd24fd None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:49.917104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-69cb820c-217a-400d-9916-4dff76fd24fd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 21:03:49.917104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:49.917104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:03:49.917104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:03:49.917104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:03:49.917104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:03:49.917104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:03:49.917104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:49.917104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 21:03:49.917104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:49.917104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:03:49.917104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:49.917104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:49.917104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:49.917104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:49.917104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:49.917104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:49.918141 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:49.918141 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:49.918141 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:49.918141 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:49.918141 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:49.918141 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:49.918141 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:03:49.918141 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:03:49.918141 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:03:49.918141 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:03:49.918141 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:49.918141 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:49.918141 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:49.918141 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:49.918141 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:49.918141 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:49.919029 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:49.919029 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:49.919029 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:49.919029 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:49.919029 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:49.919029 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:49.919029 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:49.919029 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:49.919029 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:49.919029 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:49.919029 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:03:49.919029 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:49.919029 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 27 21:03:49.919029 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:49.925151 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 458/918] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 27 21:03:49 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:49.928460 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cb820c-217a-400d-9916-4dff76fd24fd None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:49.951643 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:49.956187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81018810-f6d1-4605-98b9-0adc6417848a None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:50.004958 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3400a78-1308-4f33-b181-98f3452d6fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:50.004958 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3400a78-1308-4f33-b181-98f3452d6fad None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:50.004958 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3400a78-1308-4f33-b181-98f3452d6fad None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:50.004958 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3400a78-1308-4f33-b181-98f3452d6fad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:50.016666 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3400a78-1308-4f33-b181-98f3452d6fad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:50.018221 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3400a78-1308-4f33-b181-98f3452d6fad None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:50.040355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-b3400a78-1308-4f33-b181-98f3452d6fad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:50.040355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:50.040355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:50.040355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:50.040355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:50.040355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:50.040355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:50.040355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:50.040355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:50.040355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:50.040355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:50.040355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:50.040355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:50.040355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:50.040355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:50.041594 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:50.041594 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:50.041594 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:50.041594 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:50.041594 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:50.041594 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:50.041594 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:50.041594 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:50.041594 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:50.041594 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:50.041594 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:50.041594 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:50.041594 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:50.041594 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:50.041594 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:50.041594 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:50.041594 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:50.042701 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:50.042701 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:50.042701 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:50.042701 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:50.042701 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:50.042701 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:50.042701 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:50.042701 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:50.046631 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 461/919] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:50.047692 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3400a78-1308-4f33-b181-98f3452d6fad None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:50.078382 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cb820c-217a-400d-9916-4dff76fd24fd None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:50.121001 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d844ae44-d7d2-4c59-8241-93d94f35cc5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:50.122214 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844ae44-d7d2-4c59-8241-93d94f35cc5d None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:50.123081 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844ae44-d7d2-4c59-8241-93d94f35cc5d None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:50.123829 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d844ae44-d7d2-4c59-8241-93d94f35cc5d None admin] PATH_INFO: `/v3/services` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:50.125019 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d844ae44-d7d2-4c59-8241-93d94f35cc5d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:50.125896 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d844ae44-d7d2-4c59-8241-93d94f35cc5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:50.127956 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d844ae44-d7d2-4c59-8241-93d94f35cc5d None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:50.135595 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 459/920] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 27 21:03:49 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:50.145042 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d844ae44-d7d2-4c59-8241-93d94f35cc5d None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:50.160592 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:50.172330 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3400a78-1308-4f33-b181-98f3452d6fad None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:50.208968 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d96f3-2e9e-4430-8ea7-1cf2ab41dc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:50.209642 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d96f3-2e9e-4430-8ea7-1cf2ab41dc19 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:50.209887 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d96f3-2e9e-4430-8ea7-1cf2ab41dc19 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:50.210158 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d96f3-2e9e-4430-8ea7-1cf2ab41dc19 None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:50.210945 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582d96f3-2e9e-4430-8ea7-1cf2ab41dc19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:50.211250 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582d96f3-2e9e-4430-8ea7-1cf2ab41dc19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:50.213009 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582d96f3-2e9e-4430-8ea7-1cf2ab41dc19 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:50.231274 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 462/921] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:50.232276 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d96f3-2e9e-4430-8ea7-1cf2ab41dc19 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:50.254574 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:50.295633 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d844ae44-d7d2-4c59-8241-93d94f35cc5d None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:50.338955 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be3a8f-12b7-4c34-bdde-90aec502e917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:50.339563 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be3a8f-12b7-4c34-bdde-90aec502e917 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:50.339847 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be3a8f-12b7-4c34-bdde-90aec502e917 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:50.340132 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be3a8f-12b7-4c34-bdde-90aec502e917 None admin] PATH_INFO: `/v3/services` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:50.340849 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be3a8f-12b7-4c34-bdde-90aec502e917 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:50.341211 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be3a8f-12b7-4c34-bdde-90aec502e917 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:50.342600 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be3a8f-12b7-4c34-bdde-90aec502e917 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:50.352645 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 460/922] 10.222.0.13 () {56 vars in 1114 bytes} [Mon Jun 27 21:03:50 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:50.353585 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be3a8f-12b7-4c34-bdde-90aec502e917 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:50.360402 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d96f3-2e9e-4430-8ea7-1cf2ab41dc19 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:50.398597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd6275-5cf3-4c77-85b5-979d3211edfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:50.399592 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd6275-5cf3-4c77-85b5-979d3211edfa None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:50.400043 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd6275-5cf3-4c77-85b5-979d3211edfa None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:50.400500 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd6275-5cf3-4c77-85b5-979d3211edfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:50.402648 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dd6275-5cf3-4c77-85b5-979d3211edfa None admin] RBAC: Adding query filter params (role.id=b3dcc5de8383479d995a9bc38d8a8ef2, scope.project.id=334438c929c44e489e9a3b93e6d37231, user.id=f67110d591a94537a3df6a2d835370f9) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:50.403146 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dd6275-5cf3-4c77-85b5-979d3211edfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:50.404785 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dd6275-5cf3-4c77-85b5-979d3211edfa None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:50.411548 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 463/923] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:03:50 2022] GET /identity/v3/role_assignments?user.id=f67110d591a94537a3df6a2d835370f9&scope.project.id=334438c929c44e489e9a3b93e6d37231&role.id=b3dcc5de8383479d995a9bc38d8a8ef2 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:50.445699 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:50.506451 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be3a8f-12b7-4c34-bdde-90aec502e917 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:50.570366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129e32a5-2e10-4891-9744-8d705831ea7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:50.571910 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e32a5-2e10-4891-9744-8d705831ea7b None admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:50.572715 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e32a5-2e10-4891-9744-8d705831ea7b None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:50.573423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e32a5-2e10-4891-9744-8d705831ea7b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:50.575266 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129e32a5-2e10-4891-9744-8d705831ea7b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:50.578091 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129e32a5-2e10-4891-9744-8d705831ea7b None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:50.614176 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 461/924] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Jun 27 21:03:50 2022] POST /identity/v3/endpoints => generated 346 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:03:50.992258 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4508864-c580-47b3-84d8-0c082e7a98f2 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:50.992677 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4508864-c580-47b3-84d8-0c082e7a98f2 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:50.992848 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4508864-c580-47b3-84d8-0c082e7a98f2 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:50.994640 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 464/925] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:51.000336 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff14b94-d357-4d93-a891-2739d904f330 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:51.000630 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff14b94-d357-4d93-a891-2739d904f330 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:51.000907 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff14b94-d357-4d93-a891-2739d904f330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:51.020587 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-bff14b94-d357-4d93-a891-2739d904f330 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:51.021828 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-bff14b94-d357-4d93-a891-2739d904f330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:51.098699 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 462/926] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:03:51.121425 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4508864-c580-47b3-84d8-0c082e7a98f2 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:51.209650 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:51.228609 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4508864-c580-47b3-84d8-0c082e7a98f2 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:51.267836 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b64828-1576-4013-95f6-61d60d0f108b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:51.268987 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b64828-1576-4013-95f6-61d60d0f108b None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:51.269464 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b64828-1576-4013-95f6-61d60d0f108b None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:51.269949 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b64828-1576-4013-95f6-61d60d0f108b None admin] PATH_INFO: `/v3/services/compute` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:51.270792 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b64828-1576-4013-95f6-61d60d0f108b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:51.272364 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b64828-1576-4013-95f6-61d60d0f108b None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:51.284124 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-88b64828-1576-4013-95f6-61d60d0f108b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 21:03:51.284124 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:51.284124 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:03:51.284124 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:03:51.284124 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:03:51.284124 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:03:51.284124 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:03:51.284124 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:51.284124 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 21:03:51.284124 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:51.284124 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:03:51.284124 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:51.284124 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:51.284124 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:51.284124 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:51.284124 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:51.284124 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:51.285144 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:51.285144 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:51.285144 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:51.285144 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:51.285144 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:51.285144 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:51.285144 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:03:51.285144 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:03:51.285144 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:03:51.285144 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:03:51.285144 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:51.285144 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:51.285144 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:51.285144 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:51.285144 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:51.285144 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:51.285988 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:51.285988 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:51.285988 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:51.285988 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:51.285988 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:51.285988 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:51.285988 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:51.285988 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:51.285988 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:51.285988 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:51.285988 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:03:51.285988 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:51.285988 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 21:03:51.285988 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:51.288792 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 465/927] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:51.295184 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff14b94-d357-4d93-a891-2739d904f330 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:51.350711 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:51.399482 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff14b94-d357-4d93-a891-2739d904f330 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:51.434551 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5082c4cc-a40e-487a-a6e6-7bce79b4ccf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:51.435474 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082c4cc-a40e-487a-a6e6-7bce79b4ccf2 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:51.436097 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082c4cc-a40e-487a-a6e6-7bce79b4ccf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:51.436510 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082c4cc-a40e-487a-a6e6-7bce79b4ccf2 None admin] PATH_INFO: `/v3/services` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:51.437397 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5082c4cc-a40e-487a-a6e6-7bce79b4ccf2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:51.437841 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5082c4cc-a40e-487a-a6e6-7bce79b4ccf2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:51.439362 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5082c4cc-a40e-487a-a6e6-7bce79b4ccf2 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:51.445073 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 463/928] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Jun 27 21:03:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:51.454112 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b64828-1576-4013-95f6-61d60d0f108b None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:51.536224 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7070d5-2d84-498f-baed-2d29df8155a3 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:51.536224 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7070d5-2d84-498f-baed-2d29df8155a3 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:51.536504 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7070d5-2d84-498f-baed-2d29df8155a3 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:51.537979 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 464/929] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:51.543125 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a177acd-811d-4d3d-a3ec-c39c7e1d4f95 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:51.543412 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a177acd-811d-4d3d-a3ec-c39c7e1d4f95 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:51.543698 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a177acd-811d-4d3d-a3ec-c39c7e1d4f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:51.561046 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-1a177acd-811d-4d3d-a3ec-c39c7e1d4f95 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:51.562273 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-1a177acd-811d-4d3d-a3ec-c39c7e1d4f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:51.564709 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b64828-1576-4013-95f6-61d60d0f108b None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:51.602188 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca302c3-3fc9-4204-b0b5-2f0e34229560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:51.602860 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca302c3-3fc9-4204-b0b5-2f0e34229560 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:51.603459 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca302c3-3fc9-4204-b0b5-2f0e34229560 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:51.603771 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca302c3-3fc9-4204-b0b5-2f0e34229560 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:51.604622 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca302c3-3fc9-4204-b0b5-2f0e34229560 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:51.605317 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca302c3-3fc9-4204-b0b5-2f0e34229560 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:51.606753 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca302c3-3fc9-4204-b0b5-2f0e34229560 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:51.612962 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 466/930] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Jun 27 21:03:51 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:51.651834 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 465/931] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:03:51.674436 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca302c3-3fc9-4204-b0b5-2f0e34229560 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:51.691582 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:51.794842 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca302c3-3fc9-4204-b0b5-2f0e34229560 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:51.833333 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9991a2fe-ab2d-4361-91bf-40494beb09c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:51.834198 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991a2fe-ab2d-4361-91bf-40494beb09c4 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:51.834474 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991a2fe-ab2d-4361-91bf-40494beb09c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:51.834774 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991a2fe-ab2d-4361-91bf-40494beb09c4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:51.835436 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9991a2fe-ab2d-4361-91bf-40494beb09c4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:51.836809 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9991a2fe-ab2d-4361-91bf-40494beb09c4 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:51.849766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-9991a2fe-ab2d-4361-91bf-40494beb09c4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 21:03:51.849766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:51.849766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:03:51.849766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:03:51.849766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:03:51.849766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:03:51.849766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:03:51.849766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:51.849766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 21:03:51.849766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:51.849766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:03:51.849766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:51.849766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:51.849766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:51.849766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:51.849766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:51.849766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:51.850837 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:51.850837 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:51.850837 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:51.850837 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:51.850837 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:51.850837 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:51.850837 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:03:51.850837 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:03:51.850837 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:03:51.850837 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:03:51.850837 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:51.850837 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:51.850837 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:51.850837 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:51.850837 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:51.850837 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:51.852317 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:51.852317 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:51.852317 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:51.852317 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:51.852317 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:51.852317 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:51.852317 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:51.852317 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:51.852317 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:51.852317 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:51.852317 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:03:51.852317 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:51.852317 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 21:03:51.852317 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:51.852317 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 467/932] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Jun 27 21:03:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:51.857703 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a177acd-811d-4d3d-a3ec-c39c7e1d4f95 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:51.914772 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:51.977872 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a177acd-811d-4d3d-a3ec-c39c7e1d4f95 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:52.020343 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f25480-d0e1-4e5a-9ef5-f26ba36d9dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:52.024957 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f25480-d0e1-4e5a-9ef5-f26ba36d9dc7 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:52.024957 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f25480-d0e1-4e5a-9ef5-f26ba36d9dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:52.024957 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f25480-d0e1-4e5a-9ef5-f26ba36d9dc7 None admin] PATH_INFO: `/v3/services` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:52.024957 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f25480-d0e1-4e5a-9ef5-f26ba36d9dc7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:52.024957 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f25480-d0e1-4e5a-9ef5-f26ba36d9dc7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:52.024957 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f25480-d0e1-4e5a-9ef5-f26ba36d9dc7 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:52.033325 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 466/933] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Jun 27 21:03:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:52.039043 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9991a2fe-ab2d-4361-91bf-40494beb09c4 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:52.138188 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:52.165415 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9991a2fe-ab2d-4361-91bf-40494beb09c4 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:52.209639 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20439e72-8eae-43c4-b1ab-b7664815ac3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:52.210721 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20439e72-8eae-43c4-b1ab-b7664815ac3e None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:52.211649 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20439e72-8eae-43c4-b1ab-b7664815ac3e None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:52.212117 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20439e72-8eae-43c4-b1ab-b7664815ac3e None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:52.213320 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20439e72-8eae-43c4-b1ab-b7664815ac3e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:52.214536 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20439e72-8eae-43c4-b1ab-b7664815ac3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:52.216139 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20439e72-8eae-43c4-b1ab-b7664815ac3e None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:52.222470 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 468/934] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Jun 27 21:03:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:52.228599 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f25480-d0e1-4e5a-9ef5-f26ba36d9dc7 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:52.343125 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f25480-d0e1-4e5a-9ef5-f26ba36d9dc7 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:52.358883 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:52.384096 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9254890-83a5-4a9f-a423-9e8c1ac54804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:52.385923 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9254890-83a5-4a9f-a423-9e8c1ac54804 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:52.386480 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9254890-83a5-4a9f-a423-9e8c1ac54804 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:52.386899 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9254890-83a5-4a9f-a423-9e8c1ac54804 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:52.387869 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9254890-83a5-4a9f-a423-9e8c1ac54804 None admin] RBAC: Adding query filter params (interface=public, service_id=06948e60ee834a0c94858c9a0052ede0, region_id=RegionOne) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:52.388378 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9254890-83a5-4a9f-a423-9e8c1ac54804 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:52.389982 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9254890-83a5-4a9f-a423-9e8c1ac54804 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:52.397010 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 467/935] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Jun 27 21:03:52 2022] GET /identity/v3/endpoints?service_id=06948e60ee834a0c94858c9a0052ede0&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:52.407324 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20439e72-8eae-43c4-b1ab-b7664815ac3e None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:52.526487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20439e72-8eae-43c4-b1ab-b7664815ac3e None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:52.542458 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e24ad-6e85-4bd2-a3a0-3269359f8ff6 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:52.542742 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e24ad-6e85-4bd2-a3a0-3269359f8ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:52.543217 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e24ad-6e85-4bd2-a3a0-3269359f8ff6 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:52.544607 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 468/936] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:52.549945 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e872cd-72dc-4d8e-9ffa-5d72772fc5fc None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:52.550229 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e872cd-72dc-4d8e-9ffa-5d72772fc5fc None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:52.550746 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e872cd-72dc-4d8e-9ffa-5d72772fc5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:52.569224 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-e1e872cd-72dc-4d8e-9ffa-5d72772fc5fc None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:52.571434 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-e1e872cd-72dc-4d8e-9ffa-5d72772fc5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:52.575904 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc666467-f090-4c63-91cf-6032746fbd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:52.576960 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc666467-f090-4c63-91cf-6032746fbd08 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:52.577449 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc666467-f090-4c63-91cf-6032746fbd08 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:52.577951 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc666467-f090-4c63-91cf-6032746fbd08 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:52.578843 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc666467-f090-4c63-91cf-6032746fbd08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:52.580429 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc666467-f090-4c63-91cf-6032746fbd08 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:52.587950 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 469/937] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Jun 27 21:03:52 2022] GET /identity/v3/services => generated 1497 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:03:52.668723 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 469/938] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:03:52.694847 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc666467-f090-4c63-91cf-6032746fbd08 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:52.699576 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:52.816499 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc666467-f090-4c63-91cf-6032746fbd08 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:52.860948 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf3e67-c97a-4521-826c-cf8bf18e9a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:52.861802 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf3e67-c97a-4521-826c-cf8bf18e9a7e None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:52.862085 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf3e67-c97a-4521-826c-cf8bf18e9a7e None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:52.862358 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf3e67-c97a-4521-826c-cf8bf18e9a7e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:52.874596 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bf3e67-c97a-4521-826c-cf8bf18e9a7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:52.883008 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-34bf3e67-c97a-4521-826c-cf8bf18e9a7e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 21:03:52.883008 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:52.883008 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:52.883008 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:52.883008 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:52.883008 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:52.883008 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:52.883008 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:52.883008 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:52.883008 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:52.883008 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:52.883008 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:52.883008 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:52.883008 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:52.883008 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:52.883008 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:52.884073 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:52.884073 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:52.884073 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:52.884073 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:52.884073 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:52.884073 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:52.884073 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:52.884073 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:52.884073 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:52.884073 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:52.884073 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:52.884073 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:52.884073 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:52.884073 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:52.884073 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:52.884073 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:52.885017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:52.885017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:52.885017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:52.885017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:52.885017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:52.885017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:52.885017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:52.885017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:52.885017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:52.885017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:52.885017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 21:03:52.885017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:52.885017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 470/939] 10.222.0.13 () {56 vars in 1120 bytes} [Mon Jun 27 21:03:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:52.892856 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e872cd-72dc-4d8e-9ffa-5d72772fc5fc None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:52.897738 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf3fc9-0550-452d-ab86-3feeda091c7f None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:52.898024 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf3fc9-0550-452d-ab86-3feeda091c7f None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:52.898301 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf3fc9-0550-452d-ab86-3feeda091c7f None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:52.899918 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 471/940] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:52.906418 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b778a27-dcca-4072-af72-65e23aab087f None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:52.907062 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b778a27-dcca-4072-af72-65e23aab087f None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:52.907540 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b778a27-dcca-4072-af72-65e23aab087f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:52.925389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-7b778a27-dcca-4072-af72-65e23aab087f None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:52.927188 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-7b778a27-dcca-4072-af72-65e23aab087f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:52.990859 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:53.007094 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 472/941] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:03:53.029171 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b778a27-dcca-4072-af72-65e23aab087f None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:53.035795 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e872cd-72dc-4d8e-9ffa-5d72772fc5fc None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:53.061316 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:53.073145 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cadc5bf-7770-431e-ab14-1cdb6452f159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:53.074004 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cadc5bf-7770-431e-ab14-1cdb6452f159 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:53.074387 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cadc5bf-7770-431e-ab14-1cdb6452f159 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:53.074759 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cadc5bf-7770-431e-ab14-1cdb6452f159 None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:53.075627 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cadc5bf-7770-431e-ab14-1cdb6452f159 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:53.076035 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cadc5bf-7770-431e-ab14-1cdb6452f159 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:53.077886 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cadc5bf-7770-431e-ab14-1cdb6452f159 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:53.087323 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 470/942] 10.222.0.13 () {56 vars in 1115 bytes} [Mon Jun 27 21:03:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:53.095162 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cadc5bf-7770-431e-ab14-1cdb6452f159 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:53.170887 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b778a27-dcca-4072-af72-65e23aab087f None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:53.223538 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cadc5bf-7770-431e-ab14-1cdb6452f159 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:53.226352 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05d941-0d1c-4f4a-8495-d8f4407dce41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:53.227915 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05d941-0d1c-4f4a-8495-d8f4407dce41 None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:53.228392 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05d941-0d1c-4f4a-8495-d8f4407dce41 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:53.228911 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05d941-0d1c-4f4a-8495-d8f4407dce41 None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:53.230296 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc05d941-0d1c-4f4a-8495-d8f4407dce41 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:53.232184 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc05d941-0d1c-4f4a-8495-d8f4407dce41 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:53.234060 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: WARNING keystone.server.flask.common [None req-bc05d941-0d1c-4f4a-8495-d8f4407dce41 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 27 21:03:53.255719 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.sql.core [None req-bc05d941-0d1c-4f4a-8495-d8f4407dce41 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 27 21:03:53.255719 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 27 21:03:53.255719 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [parameters: {'id': '4aefb78562714492add23a32f49f139c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 27 21:03:53.255719 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80388) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 27 21:03:53.261549 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f01afe-1607-4128-b84e-e6adcbc4c1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:53.262368 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f01afe-1607-4128-b84e-e6adcbc4c1ae None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:53.262648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f01afe-1607-4128-b84e-e6adcbc4c1ae None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:53.262922 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f01afe-1607-4128-b84e-e6adcbc4c1ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:53.275003 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f01afe-1607-4128-b84e-e6adcbc4c1ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:53.276891 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f01afe-1607-4128-b84e-e6adcbc4c1ae None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:53.282575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-bc05d941-0d1c-4f4a-8495-d8f4407dce41 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4aefb78562714492add23a32f49f139c.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4aefb78562714492add23a32f49f139c. Jun 27 21:03:53.282575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:53.282575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 27 21:03:53.282575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 27 21:03:53.282575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 27 21:03:53.282575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 27 21:03:53.282575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 27 21:03:53.282575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application result = self._query(query) Jun 27 21:03:53.282575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 27 21:03:53.282575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application conn.query(q) Jun 27 21:03:53.282575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 27 21:03:53.282575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 27 21:03:53.282575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 27 21:03:53.283929 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application result.read() Jun 27 21:03:53.283929 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 27 21:03:53.283929 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 27 21:03:53.283929 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 27 21:03:53.283929 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 27 21:03:53.283929 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 27 21:03:53.283929 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 27 21:03:53.283929 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 27 21:03:53.283929 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 27 21:03:53.283929 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 27 21:03:53.283929 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:53.283929 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 27 21:03:53.283929 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:53.283929 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:53.283929 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 27 21:03:53.283929 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 27 21:03:53.283929 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 27 21:03:53.283929 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 27 21:03:53.285483 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 27 21:03:53.285483 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application next(self.gen) Jun 27 21:03:53.285483 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 27 21:03:53.285483 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application yield resource Jun 27 21:03:53.285483 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 27 21:03:53.285483 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application next(self.gen) Jun 27 21:03:53.285483 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 27 21:03:53.285483 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self.session.rollback() Jun 27 21:03:53.285483 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 27 21:03:53.285483 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self.force_reraise() Jun 27 21:03:53.285483 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 27 21:03:53.285483 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise self.value Jun 27 21:03:53.285483 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 27 21:03:53.285483 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 27 21:03:53.285483 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 27 21:03:53.285483 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application session.commit() Jun 27 21:03:53.285483 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 27 21:03:53.286949 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 27 21:03:53.286949 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 27 21:03:53.286949 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._prepare_impl() Jun 27 21:03:53.286949 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 27 21:03:53.286949 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self.session.flush() Jun 27 21:03:53.286949 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 27 21:03:53.286949 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._flush(objects) Jun 27 21:03:53.286949 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 27 21:03:53.286949 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 27 21:03:53.286949 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 27 21:03:53.286949 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application compat.raise_( Jun 27 21:03:53.286949 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 27 21:03:53.286949 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception Jun 27 21:03:53.286949 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 27 21:03:53.286949 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application flush_context.execute() Jun 27 21:03:53.286949 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 27 21:03:53.286949 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rec.execute(self) Jun 27 21:03:53.288430 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 27 21:03:53.288430 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 27 21:03:53.288430 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 27 21:03:53.288430 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 27 21:03:53.288430 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 27 21:03:53.288430 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 27 21:03:53.288430 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 27 21:03:53.288430 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 27 21:03:53.288430 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 27 21:03:53.288430 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 27 21:03:53.288430 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 27 21:03:53.288430 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 27 21:03:53.288430 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 27 21:03:53.288430 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 27 21:03:53.288430 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 27 21:03:53.292355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 27 21:03:53.292355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 27 21:03:53.292355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception Jun 27 21:03:53.292355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 27 21:03:53.292355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 27 21:03:53.292355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 27 21:03:53.292355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 27 21:03:53.292355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 27 21:03:53.292355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application result = self._query(query) Jun 27 21:03:53.292355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 27 21:03:53.292355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application conn.query(q) Jun 27 21:03:53.292355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 27 21:03:53.292355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 27 21:03:53.292355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 27 21:03:53.292355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application result.read() Jun 27 21:03:53.292355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 27 21:03:53.292355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 27 21:03:53.295533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 27 21:03:53.295533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 27 21:03:53.295533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 27 21:03:53.295533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 27 21:03:53.295533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 27 21:03:53.295533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 27 21:03:53.295533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 27 21:03:53.295533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 27 21:03:53.295533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [parameters: {'id': '4aefb78562714492add23a32f49f139c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 27 21:03:53.295533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 27 21:03:53.295533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:53.295533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:03:53.295533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:53.295533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:53.295533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 27 21:03:53.298524 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 27 21:03:53.298524 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 27 21:03:53.298524 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 27 21:03:53.298524 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 27 21:03:53.298524 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:53.298524 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:03:53.298524 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:53.298524 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:53.298524 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:53.298524 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:53.298524 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:53.298524 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:53.298524 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:53.298524 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:53.298524 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:53.298524 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:53.298524 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:53.301531 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:53.301531 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 27 21:03:53.301531 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 27 21:03:53.301531 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:53.301531 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:53.301531 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 27 21:03:53.301531 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 27 21:03:53.301531 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4aefb78562714492add23a32f49f139c. Jun 27 21:03:53.301531 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:53.302946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-86f01afe-1607-4128-b84e-e6adcbc4c1ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:53.302946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:53.302946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:53.302946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:53.302946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:53.302946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:53.302946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:53.302946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:53.302946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:53.302946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:53.302946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:53.302946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:53.302946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:53.302946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:53.302946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:53.304806 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:53.304806 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:53.304806 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:53.304806 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:53.304806 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:53.304806 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:53.304806 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:53.304806 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:53.304806 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:53.304806 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:53.304806 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:53.304806 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:53.304806 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:53.304806 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:53.304806 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:53.304806 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:53.304806 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:53.305860 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:53.305860 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:53.305860 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:53.305860 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:53.306080 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 473/943] 10.222.0.13 () {60 vars in 1140 bytes} [Mon Jun 27 21:03:53 2022] POST /identity/v3/projects => generated 262 bytes in 274 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:53.306150 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 471/944] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:53.310970 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05d941-0d1c-4f4a-8495-d8f4407dce41 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:53.316870 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f01afe-1607-4128-b84e-e6adcbc4c1ae None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:53.359483 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:53.387891 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:53.445932 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05d941-0d1c-4f4a-8495-d8f4407dce41 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:53.484003 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69734f6a-c8c7-4b31-9143-a906473ee175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:53.485178 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69734f6a-c8c7-4b31-9143-a906473ee175 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:53.485632 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69734f6a-c8c7-4b31-9143-a906473ee175 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:53.486052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69734f6a-c8c7-4b31-9143-a906473ee175 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:53.493946 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69734f6a-c8c7-4b31-9143-a906473ee175 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:53.496054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69734f6a-c8c7-4b31-9143-a906473ee175 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:53.512179 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-69734f6a-c8c7-4b31-9143-a906473ee175 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:53.512179 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:53.512179 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:53.512179 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:53.512179 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:53.512179 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:53.512179 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:53.512179 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:53.512179 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:53.512179 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:53.512179 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:53.512179 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:53.512179 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:53.512179 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:53.512179 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:53.513338 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:53.513338 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:53.513338 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:53.513338 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:53.513338 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:53.513338 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:53.513338 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:53.513338 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:53.513338 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:53.513338 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:53.513338 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:53.513338 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:53.513338 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:53.513338 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:53.513338 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:53.513338 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:53.513338 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:53.514367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:53.514367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:53.514367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:53.514367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:53.514367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:53.514367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:53.514367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:53.514367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:53.516460 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 474/945] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:53.524842 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f01afe-1607-4128-b84e-e6adcbc4c1ae None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:53.530813 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69734f6a-c8c7-4b31-9143-a906473ee175 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:53.565828 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce3ff7-def0-4ba1-bfcc-5fbb44b0f977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:53.566870 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce3ff7-def0-4ba1-bfcc-5fbb44b0f977 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:53.567157 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce3ff7-def0-4ba1-bfcc-5fbb44b0f977 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:53.567433 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce3ff7-def0-4ba1-bfcc-5fbb44b0f977 None admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:53.568467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ce3ff7-def0-4ba1-bfcc-5fbb44b0f977 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:53.568774 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ce3ff7-def0-4ba1-bfcc-5fbb44b0f977 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:53.570744 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ce3ff7-def0-4ba1-bfcc-5fbb44b0f977 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:53.588965 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 472/946] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:53.593698 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce3ff7-def0-4ba1-bfcc-5fbb44b0f977 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:53.663597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69734f6a-c8c7-4b31-9143-a906473ee175 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:53.681590 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:53.699252 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:53.718557 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bacdec-6585-417e-8bf9-9f0aad2a6707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:53.720239 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bacdec-6585-417e-8bf9-9f0aad2a6707 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:53.720816 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bacdec-6585-417e-8bf9-9f0aad2a6707 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:53.721430 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bacdec-6585-417e-8bf9-9f0aad2a6707 None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:53.722573 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bacdec-6585-417e-8bf9-9f0aad2a6707 None admin] RBAC: Adding query filter params (name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:53.723431 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bacdec-6585-417e-8bf9-9f0aad2a6707 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:53.725349 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bacdec-6585-417e-8bf9-9f0aad2a6707 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:53.726328 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce3ff7-def0-4ba1-bfcc-5fbb44b0f977 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:53.761470 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 475/947] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Jun 27 21:03:53 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:53.773274 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3052da46-679b-4d7d-aec3-a028f76743ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:53.774460 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052da46-679b-4d7d-aec3-a028f76743ba None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:53.774882 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052da46-679b-4d7d-aec3-a028f76743ba None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:53.775319 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052da46-679b-4d7d-aec3-a028f76743ba None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:53.784776 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3052da46-679b-4d7d-aec3-a028f76743ba None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:53.786587 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3052da46-679b-4d7d-aec3-a028f76743ba None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:53.804127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-3052da46-679b-4d7d-aec3-a028f76743ba None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 21:03:53.804127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:53.804127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:53.804127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:53.804127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:53.804127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:53.804127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:53.804127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:53.804127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:53.804127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:53.804127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:53.804127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:53.804127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:53.804127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:53.804127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:53.804127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:53.805384 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:53.805384 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:53.805384 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:53.805384 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:53.805384 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:53.805384 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:53.805384 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:53.805384 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:53.805384 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:53.805384 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:53.805384 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:53.805384 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:53.805384 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:53.805384 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:53.805384 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:53.805384 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:53.805384 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:53.805384 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:53.806528 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:53.806528 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:53.806528 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:53.806528 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:53.806528 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:53.806528 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:53.806528 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:53.806528 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:53.806528 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 21:03:53.806528 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:53.809277 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 473/948] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Jun 27 21:03:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:53.816992 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bacdec-6585-417e-8bf9-9f0aad2a6707 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:53.904219 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6e01d-04e3-42bc-8aa3-82ee197f6cd0 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:53.904508 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6e01d-04e3-42bc-8aa3-82ee197f6cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:53.904794 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6e01d-04e3-42bc-8aa3-82ee197f6cd0 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:53.910572 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 474/949] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:03:53.912466 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097ede8-8d1c-40bb-baaf-a84e0fad4109 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:53.912752 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097ede8-8d1c-40bb-baaf-a84e0fad4109 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:53.913061 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097ede8-8d1c-40bb-baaf-a84e0fad4109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:53.935776 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-d097ede8-8d1c-40bb-baaf-a84e0fad4109 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:53.937857 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-d097ede8-8d1c-40bb-baaf-a84e0fad4109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:53.939009 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bacdec-6585-417e-8bf9-9f0aad2a6707 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:53.991744 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473777c7-93fe-4ee5-84c8-27bf3649f4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:53.991744 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473777c7-93fe-4ee5-84c8-27bf3649f4f7 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:53.991744 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473777c7-93fe-4ee5-84c8-27bf3649f4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:53.991744 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473777c7-93fe-4ee5-84c8-27bf3649f4f7 None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:53.992428 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473777c7-93fe-4ee5-84c8-27bf3649f4f7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:53.992428 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473777c7-93fe-4ee5-84c8-27bf3649f4f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:53.993470 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473777c7-93fe-4ee5-84c8-27bf3649f4f7 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:54.020778 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 476/950] 10.222.0.13 () {56 vars in 1149 bytes} [Mon Jun 27 21:03:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:54.030969 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473777c7-93fe-4ee5-84c8-27bf3649f4f7 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:54.048965 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 475/951] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:03:54.064211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:54.071873 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d097ede8-8d1c-40bb-baaf-a84e0fad4109 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:54.145670 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:54.153417 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473777c7-93fe-4ee5-84c8-27bf3649f4f7 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:54.184376 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d097ede8-8d1c-40bb-baaf-a84e0fad4109 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:54.191960 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2344fb-3d81-40df-978e-d686e4a7bd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:54.193295 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2344fb-3d81-40df-978e-d686e4a7bd4c None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:54.193295 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2344fb-3d81-40df-978e-d686e4a7bd4c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:54.193513 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2344fb-3d81-40df-978e-d686e4a7bd4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:54.205643 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2344fb-3d81-40df-978e-d686e4a7bd4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:54.207176 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2344fb-3d81-40df-978e-d686e4a7bd4c None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:54.222602 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6592546a-9013-4b17-907a-722555cf4620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:54.223535 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592546a-9013-4b17-907a-722555cf4620 None admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:54.223993 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592546a-9013-4b17-907a-722555cf4620 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:54.224414 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592546a-9013-4b17-907a-722555cf4620 None admin] PATH_INFO: `/v3/services` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:54.225342 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6592546a-9013-4b17-907a-722555cf4620 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:54.227241 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6592546a-9013-4b17-907a-722555cf4620 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:54.228206 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-dd2344fb-3d81-40df-978e-d686e4a7bd4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:54.228206 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:54.228206 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:54.228206 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:54.228206 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:54.228206 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:54.228206 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:54.228206 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:54.228206 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:54.228206 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:54.228206 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:54.228206 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:54.228206 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:54.228206 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:54.228206 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:54.229715 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:54.229715 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:54.229715 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:54.229715 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:54.229715 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:54.229715 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:54.229715 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:54.229715 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:54.229715 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:54.229715 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:54.229715 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:54.229715 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:54.229715 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:54.229715 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:54.229715 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:54.229715 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:54.229715 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:54.230972 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:54.230972 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:54.230972 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:54.230972 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:54.233244 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 477/952] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:54.235836 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2344fb-3d81-40df-978e-d686e4a7bd4c None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:54.253126 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 476/953] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Jun 27 21:03:54 2022] POST /identity/v3/services => generated 242 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:03:54.343526 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:54.369049 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2344fb-3d81-40df-978e-d686e4a7bd4c None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:54.420102 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1381cfed-dee3-4f14-90bb-56245e6eaa46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:54.420823 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381cfed-dee3-4f14-90bb-56245e6eaa46 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:54.421134 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381cfed-dee3-4f14-90bb-56245e6eaa46 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:54.421411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381cfed-dee3-4f14-90bb-56245e6eaa46 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:54.422156 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1381cfed-dee3-4f14-90bb-56245e6eaa46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:54.422451 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1381cfed-dee3-4f14-90bb-56245e6eaa46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:54.423845 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1381cfed-dee3-4f14-90bb-56245e6eaa46 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:54.437625 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 478/954] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:54.443332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6592546a-9013-4b17-907a-722555cf4620 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:54.575052 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6592546a-9013-4b17-907a-722555cf4620 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:54.609499 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:54.624140 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ccdbbb-1026-4cc9-9401-57836a707e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:54.625297 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ccdbbb-1026-4cc9-9401-57836a707e10 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:54.625603 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ccdbbb-1026-4cc9-9401-57836a707e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:54.625926 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ccdbbb-1026-4cc9-9401-57836a707e10 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:54.634355 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ccdbbb-1026-4cc9-9401-57836a707e10 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:54.636664 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ccdbbb-1026-4cc9-9401-57836a707e10 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:54.653108 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-11ccdbbb-1026-4cc9-9401-57836a707e10 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:54.653108 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:54.653108 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:54.653108 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:54.653108 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:54.653108 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:54.653108 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:54.653108 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:54.653108 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:54.653108 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:54.653108 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:54.653108 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:54.653108 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:54.653108 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:54.653108 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:54.654150 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:54.654150 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:54.654150 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:54.654150 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:54.654150 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:54.654150 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:54.654150 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:54.654150 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:54.654150 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:54.654150 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:54.654150 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:54.654150 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:54.654150 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:54.654150 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:54.654150 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:54.654150 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:54.654150 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:54.655159 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:54.655159 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:54.655159 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:54.655159 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:54.655159 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:54.655159 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:54.655159 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:54.655159 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:54.655159 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 477/955] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:54.662252 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1381cfed-dee3-4f14-90bb-56245e6eaa46 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:54.777438 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1381cfed-dee3-4f14-90bb-56245e6eaa46 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:54.814006 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1c98ad-d60f-4e0c-bce8-1f58a620899e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:54.814724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c98ad-d60f-4e0c-bce8-1f58a620899e None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:54.815003 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c98ad-d60f-4e0c-bce8-1f58a620899e None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:54.815271 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1c98ad-d60f-4e0c-bce8-1f58a620899e None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:54.816065 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1c98ad-d60f-4e0c-bce8-1f58a620899e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:54.816366 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1c98ad-d60f-4e0c-bce8-1f58a620899e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:54.817968 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1c98ad-d60f-4e0c-bce8-1f58a620899e None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:54.820780 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:54.833881 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 479/956] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:54.840330 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ccdbbb-1026-4cc9-9401-57836a707e10 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:54.893083 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931bd0c0-12fd-4e1d-9592-3c91926b3b4e None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:54.893265 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931bd0c0-12fd-4e1d-9592-3c91926b3b4e None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:54.893543 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931bd0c0-12fd-4e1d-9592-3c91926b3b4e None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:54.895201 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 480/957] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:03:54.901864 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda8ca3-1124-4102-a47d-40b008bca13d None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:54.902168 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda8ca3-1124-4102-a47d-40b008bca13d None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:54.902410 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda8ca3-1124-4102-a47d-40b008bca13d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:54.922416 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-8eda8ca3-1124-4102-a47d-40b008bca13d None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:54.924082 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-8eda8ca3-1124-4102-a47d-40b008bca13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:54.988536 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ccdbbb-1026-4cc9-9401-57836a707e10 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:55.008572 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 481/958] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:54 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:03:55.037818 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda8ca3-1124-4102-a47d-40b008bca13d None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:55.043933 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e4a131-666e-4dcb-ab40-3eadd034bf9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:55.045311 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4a131-666e-4dcb-ab40-3eadd034bf9c None admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:55.045818 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4a131-666e-4dcb-ab40-3eadd034bf9c None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:55.046291 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4a131-666e-4dcb-ab40-3eadd034bf9c None admin] PATH_INFO: `/v3/projects/334438c929c44e489e9a3b93e6d37231/users/f67110d591a94537a3df6a2d835370f9/roles/b3dcc5de8383479d995a9bc38d8a8ef2` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:55.049371 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e4a131-666e-4dcb-ab40-3eadd034bf9c None admin] RBAC: Authorizing `identity:create_grant(project_id=334438c929c44e489e9a3b93e6d37231, user_id=f67110d591a94537a3df6a2d835370f9, role_id=b3dcc5de8383479d995a9bc38d8a8ef2)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:55.051921 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e4a131-666e-4dcb-ab40-3eadd034bf9c None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:55.084806 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 478/959] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 27 21:03:54 2022] PUT /identity/v3/projects/334438c929c44e489e9a3b93e6d37231/users/f67110d591a94537a3df6a2d835370f9/roles/b3dcc5de8383479d995a9bc38d8a8ef2 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:03:55.180542 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda8ca3-1124-4102-a47d-40b008bca13d None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:55.211173 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:55.221601 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6414d-70b2-4c61-8c3c-51a19fb33f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:55.222442 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6414d-70b2-4c61-8c3c-51a19fb33f2b None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:55.222764 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6414d-70b2-4c61-8c3c-51a19fb33f2b None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:55.223037 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6414d-70b2-4c61-8c3c-51a19fb33f2b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:55.223704 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee6414d-70b2-4c61-8c3c-51a19fb33f2b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:55.225128 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee6414d-70b2-4c61-8c3c-51a19fb33f2b None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:55.239147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-dee6414d-70b2-4c61-8c3c-51a19fb33f2b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 21:03:55.239147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:55.239147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:03:55.239147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:03:55.239147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:03:55.239147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:03:55.239147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:03:55.239147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:55.239147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 21:03:55.239147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:55.239147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:03:55.239147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:55.239147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:55.239147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:55.239147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:55.239147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:55.239147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:55.240756 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:55.240756 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:55.240756 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:55.240756 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:55.240756 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:55.240756 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:55.240756 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:03:55.240756 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:03:55.240756 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:03:55.240756 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:03:55.240756 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:55.240756 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:55.240756 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:55.240756 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:55.240756 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:55.240756 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:55.241751 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:55.241751 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:55.241751 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:55.241751 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:55.241751 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:55.241751 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:55.241751 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:55.241751 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:55.241751 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:55.241751 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:55.241751 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:03:55.241751 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:55.241751 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 21:03:55.241751 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:55.241751 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 482/960] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Jun 27 21:03:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:55.247243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e4a131-666e-4dcb-ab40-3eadd034bf9c None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:55.314631 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:55.368092 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e4a131-666e-4dcb-ab40-3eadd034bf9c None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:55.406069 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b355470f-852f-463d-ab39-441ec57b9783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:55.406778 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b355470f-852f-463d-ab39-441ec57b9783 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:55.407093 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b355470f-852f-463d-ab39-441ec57b9783 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:55.407371 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b355470f-852f-463d-ab39-441ec57b9783 None admin] PATH_INFO: `/v3/services` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:55.408588 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b355470f-852f-463d-ab39-441ec57b9783 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:55.409090 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b355470f-852f-463d-ab39-441ec57b9783 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:55.410482 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b355470f-852f-463d-ab39-441ec57b9783 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:55.417125 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 479/961] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Jun 27 21:03:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:55.423543 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6414d-70b2-4c61-8c3c-51a19fb33f2b None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:55.539056 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6414d-70b2-4c61-8c3c-51a19fb33f2b None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:55.582886 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b524fd-3fcb-4a94-9453-e7c5d3aec466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:55.583913 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b524fd-3fcb-4a94-9453-e7c5d3aec466 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:55.584416 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b524fd-3fcb-4a94-9453-e7c5d3aec466 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:55.584889 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b524fd-3fcb-4a94-9453-e7c5d3aec466 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:55.585831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b524fd-3fcb-4a94-9453-e7c5d3aec466 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:55.586331 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b524fd-3fcb-4a94-9453-e7c5d3aec466 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:55.587897 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b524fd-3fcb-4a94-9453-e7c5d3aec466 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:55.594290 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 483/962] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Jun 27 21:03:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:55.600329 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b355470f-852f-463d-ab39-441ec57b9783 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:55.711995 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b355470f-852f-463d-ab39-441ec57b9783 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:55.729373 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:55.751216 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc341e08-172b-475b-ab86-d64c1289a79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:55.752290 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc341e08-172b-475b-ab86-d64c1289a79c None admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:55.752758 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc341e08-172b-475b-ab86-d64c1289a79c None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:55.753218 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc341e08-172b-475b-ab86-d64c1289a79c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:55.754002 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc341e08-172b-475b-ab86-d64c1289a79c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:55.755388 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc341e08-172b-475b-ab86-d64c1289a79c None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:55.780680 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 480/963] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Jun 27 21:03:55 2022] POST /identity/v3/endpoints => generated 349 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:03:56.105660 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf57ae-d97f-438c-a08c-6ff7df384f66 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:56.106024 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf57ae-d97f-438c-a08c-6ff7df384f66 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:56.106208 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf57ae-d97f-438c-a08c-6ff7df384f66 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:56.107881 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 484/964] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:56.113601 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16efc112-57f4-40c1-9bbe-e125f35cd370 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:56.113888 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16efc112-57f4-40c1-9bbe-e125f35cd370 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:56.114164 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16efc112-57f4-40c1-9bbe-e125f35cd370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:56.132553 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-16efc112-57f4-40c1-9bbe-e125f35cd370 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:56.133795 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-16efc112-57f4-40c1-9bbe-e125f35cd370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:56.216690 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 481/965] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:56 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:03:56.247602 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf57ae-d97f-438c-a08c-6ff7df384f66 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:56.329656 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:56.380903 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf57ae-d97f-438c-a08c-6ff7df384f66 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:56.421497 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b239d-e6dc-4af9-adf1-3542c5c381b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:56.422176 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b239d-e6dc-4af9-adf1-3542c5c381b5 None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:56.422452 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b239d-e6dc-4af9-adf1-3542c5c381b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:56.422718 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b239d-e6dc-4af9-adf1-3542c5c381b5 None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:56.423431 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10b239d-e6dc-4af9-adf1-3542c5c381b5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:56.424899 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10b239d-e6dc-4af9-adf1-3542c5c381b5 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:56.442246 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.sql.core [None req-d10b239d-e6dc-4af9-adf1-3542c5c381b5 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 27 21:03:56.442246 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 27 21:03:56.442246 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [parameters: {'user_id': '18622c5646b5496e8669ca7dd6bde7db', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 27 21:03:56.442246 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80388) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 27 21:03:56.450632 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-d10b239d-e6dc-4af9-adf1-3542c5c381b5 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 27 21:03:56.450632 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:56.450632 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 27 21:03:56.450632 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 27 21:03:56.450632 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 27 21:03:56.450632 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 27 21:03:56.450632 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 27 21:03:56.450632 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application result = self._query(query) Jun 27 21:03:56.450632 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 27 21:03:56.450632 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application conn.query(q) Jun 27 21:03:56.450632 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 27 21:03:56.450632 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 27 21:03:56.450632 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 27 21:03:56.450632 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application result.read() Jun 27 21:03:56.450632 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 27 21:03:56.451514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 27 21:03:56.451514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 27 21:03:56.451514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 27 21:03:56.451514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 27 21:03:56.451514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 27 21:03:56.451514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 27 21:03:56.451514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 27 21:03:56.451514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 27 21:03:56.451514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:56.451514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 27 21:03:56.451514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:56.451514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:56.451514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 27 21:03:56.451514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 27 21:03:56.451514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 27 21:03:56.451514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 27 21:03:56.451514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 27 21:03:56.451514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application next(self.gen) Jun 27 21:03:56.452466 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 27 21:03:56.452466 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application yield resource Jun 27 21:03:56.452466 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 27 21:03:56.452466 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application next(self.gen) Jun 27 21:03:56.452466 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 27 21:03:56.452466 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self.session.rollback() Jun 27 21:03:56.452466 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 27 21:03:56.452466 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self.force_reraise() Jun 27 21:03:56.452466 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 27 21:03:56.452466 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise self.value Jun 27 21:03:56.452466 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 27 21:03:56.452466 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 27 21:03:56.452466 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 27 21:03:56.452466 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application session.commit() Jun 27 21:03:56.452466 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 27 21:03:56.452466 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 27 21:03:56.452466 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 27 21:03:56.453406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._prepare_impl() Jun 27 21:03:56.453406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 27 21:03:56.453406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self.session.flush() Jun 27 21:03:56.453406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 27 21:03:56.453406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._flush(objects) Jun 27 21:03:56.453406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 27 21:03:56.453406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 27 21:03:56.453406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 27 21:03:56.453406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application compat.raise_( Jun 27 21:03:56.453406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 27 21:03:56.453406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception Jun 27 21:03:56.453406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 27 21:03:56.453406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application flush_context.execute() Jun 27 21:03:56.453406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 27 21:03:56.453406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rec.execute(self) Jun 27 21:03:56.453406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 27 21:03:56.453406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 27 21:03:56.455946 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 27 21:03:56.455946 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 27 21:03:56.455946 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 27 21:03:56.455946 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 27 21:03:56.455946 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 27 21:03:56.455946 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 27 21:03:56.455946 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 27 21:03:56.455946 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 27 21:03:56.455946 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 27 21:03:56.455946 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 27 21:03:56.455946 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 27 21:03:56.455946 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 27 21:03:56.455946 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 27 21:03:56.455946 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 27 21:03:56.455946 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 27 21:03:56.455946 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception Jun 27 21:03:56.458170 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 27 21:03:56.458170 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 27 21:03:56.458170 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 27 21:03:56.458170 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 27 21:03:56.458170 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 27 21:03:56.458170 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application result = self._query(query) Jun 27 21:03:56.458170 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 27 21:03:56.458170 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application conn.query(q) Jun 27 21:03:56.458170 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 27 21:03:56.458170 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 27 21:03:56.458170 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 27 21:03:56.458170 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application result.read() Jun 27 21:03:56.458170 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 27 21:03:56.458170 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 27 21:03:56.458170 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 27 21:03:56.458170 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 27 21:03:56.460166 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 27 21:03:56.460166 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 27 21:03:56.460166 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 27 21:03:56.460166 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 27 21:03:56.460166 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 27 21:03:56.460166 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 27 21:03:56.460166 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [parameters: {'user_id': '18622c5646b5496e8669ca7dd6bde7db', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 27 21:03:56.460166 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 27 21:03:56.460166 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:56.460166 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:03:56.460166 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:56.460166 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:56.460166 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:56.460166 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:56.460166 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:56.470822 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:56.470822 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:56.470822 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:56.470822 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:56.470822 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:56.470822 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:56.470822 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:56.470822 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 27 21:03:56.470822 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 27 21:03:56.470822 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:56.470822 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:56.470822 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:56.470822 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:56.470822 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:56.470822 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:56.470822 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 27 21:03:56.470822 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 27 21:03:56.473105 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 27 21:03:56.473105 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 27 21:03:56.473105 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 27 21:03:56.473105 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 27 21:03:56.473105 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 27 21:03:56.473105 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:56.473105 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 485/966] 10.222.0.13 () {60 vars in 1131 bytes} [Mon Jun 27 21:03:56 2022] POST /identity/v3/users => generated 160 bytes in 205 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:56.474159 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16efc112-57f4-40c1-9bbe-e125f35cd370 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:56.474348 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326050d8-fe47-485f-a963-3ed56572462f None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:56.474348 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326050d8-fe47-485f-a963-3ed56572462f None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:56.474348 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326050d8-fe47-485f-a963-3ed56572462f None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:56.476992 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 486/967] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:56.482829 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df6908-1bf9-42ea-9fea-80690d9b9661 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:56.483112 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df6908-1bf9-42ea-9fea-80690d9b9661 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:56.483389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df6908-1bf9-42ea-9fea-80690d9b9661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:56.501982 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-53df6908-1bf9-42ea-9fea-80690d9b9661 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:56.503706 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-53df6908-1bf9-42ea-9fea-80690d9b9661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:56.594212 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:56.601428 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 487/968] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:56 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:03:56.609156 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16efc112-57f4-40c1-9bbe-e125f35cd370 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:56.622422 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df6908-1bf9-42ea-9fea-80690d9b9661 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:56.659012 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0173d6-37ff-43db-a1b5-c5dc6197b9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:56.660005 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0173d6-37ff-43db-a1b5-c5dc6197b9ef None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:56.660291 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0173d6-37ff-43db-a1b5-c5dc6197b9ef None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:56.660571 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0173d6-37ff-43db-a1b5-c5dc6197b9ef None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:56.671171 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0173d6-37ff-43db-a1b5-c5dc6197b9ef None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:56.673141 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0173d6-37ff-43db-a1b5-c5dc6197b9ef None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:56.678546 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:56.695581 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-1d0173d6-37ff-43db-a1b5-c5dc6197b9ef None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 21:03:56.695581 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:56.695581 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:56.695581 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:56.695581 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:56.695581 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:56.695581 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:56.695581 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:56.695581 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:56.695581 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:56.695581 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:56.695581 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:56.695581 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:56.695581 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:56.695581 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:56.695581 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:56.696637 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:56.696637 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:56.696637 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:56.696637 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:56.696637 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:56.696637 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:56.696637 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:56.696637 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:56.696637 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:56.696637 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:56.696637 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:56.696637 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:56.696637 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:56.696637 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:56.696637 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:56.696637 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:56.696637 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:56.696637 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:56.697739 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:56.697739 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:56.697739 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:56.697739 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:56.697739 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:56.697739 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:56.697739 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:56.697739 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:56.697739 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 27 21:03:56.697739 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:56.701187 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 482/969] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:03:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:56.703593 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0173d6-37ff-43db-a1b5-c5dc6197b9ef None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:56.756040 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df6908-1bf9-42ea-9fea-80690d9b9661 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:56.798700 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882bbafa-a487-4e79-9949-1275d6616f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:56.799901 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bbafa-a487-4e79-9949-1275d6616f74 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:56.800433 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bbafa-a487-4e79-9949-1275d6616f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:56.800965 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bbafa-a487-4e79-9949-1275d6616f74 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:56.801877 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882bbafa-a487-4e79-9949-1275d6616f74 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:56.803646 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882bbafa-a487-4e79-9949-1275d6616f74 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:56.816595 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-882bbafa-a487-4e79-9949-1275d6616f74 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 21:03:56.816595 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:56.816595 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:03:56.816595 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:03:56.816595 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:03:56.816595 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:03:56.816595 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:03:56.816595 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:56.816595 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 21:03:56.816595 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:56.816595 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:03:56.816595 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:56.816595 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:56.816595 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:56.816595 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:56.816595 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:56.816595 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:56.817729 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:56.817729 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:56.817729 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:56.817729 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:56.817729 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:56.817729 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:56.817729 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:03:56.817729 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:03:56.817729 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:03:56.817729 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:03:56.817729 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:56.817729 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:56.817729 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:56.817729 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:56.817729 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:56.817729 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:56.818593 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:56.818593 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:56.818593 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:56.818593 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:56.818593 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:56.818593 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:56.818593 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:56.818593 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:56.818593 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:56.818593 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:56.818593 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:03:56.818593 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:56.818593 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 21:03:56.818593 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:56.826565 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0173d6-37ff-43db-a1b5-c5dc6197b9ef None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:56.827020 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 488/970] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:56.828636 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882bbafa-a487-4e79-9949-1275d6616f74 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:56.869447 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed948f-ed0e-48a3-a553-3131f01990c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:56.871139 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed948f-ed0e-48a3-a553-3131f01990c4 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:56.871735 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed948f-ed0e-48a3-a553-3131f01990c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:56.872278 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed948f-ed0e-48a3-a553-3131f01990c4 None admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:56.873505 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed948f-ed0e-48a3-a553-3131f01990c4 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:56.874059 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed948f-ed0e-48a3-a553-3131f01990c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:56.875994 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed948f-ed0e-48a3-a553-3131f01990c4 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:56.920386 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 483/971] 10.222.0.13 () {56 vars in 1101 bytes} [Mon Jun 27 21:03:56 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:56.956010 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882bbafa-a487-4e79-9949-1275d6616f74 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:56.972375 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:57.012823 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215517f4-f970-4dd9-8020-da5737f1baa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:57.013581 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215517f4-f970-4dd9-8020-da5737f1baa9 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:57.013866 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215517f4-f970-4dd9-8020-da5737f1baa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:57.014144 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215517f4-f970-4dd9-8020-da5737f1baa9 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:57.014877 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215517f4-f970-4dd9-8020-da5737f1baa9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:57.015183 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215517f4-f970-4dd9-8020-da5737f1baa9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:57.016604 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215517f4-f970-4dd9-8020-da5737f1baa9 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:57.023039 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 489/972] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Jun 27 21:03:56 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:57.029056 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed948f-ed0e-48a3-a553-3131f01990c4 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:57.054447 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:57.163340 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed948f-ed0e-48a3-a553-3131f01990c4 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:57.221628 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8da22a-d2a6-4361-81b6-691147b71875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:57.222316 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8da22a-d2a6-4361-81b6-691147b71875 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:57.222597 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8da22a-d2a6-4361-81b6-691147b71875 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:57.222883 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8da22a-d2a6-4361-81b6-691147b71875 None admin] PATH_INFO: `/v3/services` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:57.223647 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8da22a-d2a6-4361-81b6-691147b71875 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:57.223956 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8da22a-d2a6-4361-81b6-691147b71875 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:57.225395 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8da22a-d2a6-4361-81b6-691147b71875 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:57.232997 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 484/973] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Jun 27 21:03:57 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:57.238233 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215517f4-f970-4dd9-8020-da5737f1baa9 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:57.269383 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76104d3-6658-48bd-8f7f-8da567b263b1 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:57.269989 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76104d3-6658-48bd-8f7f-8da567b263b1 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:57.270297 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76104d3-6658-48bd-8f7f-8da567b263b1 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:57.271712 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 485/974] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:03:57.277676 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d98fd4-5b21-4196-aa96-762b087957be None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:57.277961 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d98fd4-5b21-4196-aa96-762b087957be None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:57.278206 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d98fd4-5b21-4196-aa96-762b087957be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:57.297240 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-41d98fd4-5b21-4196-aa96-762b087957be None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:57.299071 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-41d98fd4-5b21-4196-aa96-762b087957be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:57.358113 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215517f4-f970-4dd9-8020-da5737f1baa9 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:57.389177 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 486/975] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:57 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:03:57.398878 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28159588-508b-4674-a97d-afc7deff0670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:57.399544 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28159588-508b-4674-a97d-afc7deff0670 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:57.399832 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28159588-508b-4674-a97d-afc7deff0670 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:57.400107 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28159588-508b-4674-a97d-afc7deff0670 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:57.401088 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28159588-508b-4674-a97d-afc7deff0670 None admin] RBAC: Adding query filter params (interface=public, service_id=9e1c9bce00754a31931152a0f6b023d1, region_id=RegionOne) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:57.401504 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28159588-508b-4674-a97d-afc7deff0670 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:57.402720 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28159588-508b-4674-a97d-afc7deff0670 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:57.413789 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 490/976] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Jun 27 21:03:57 2022] GET /identity/v3/endpoints?service_id=9e1c9bce00754a31931152a0f6b023d1&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:57.413978 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d98fd4-5b21-4196-aa96-762b087957be None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:57.419758 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28159588-508b-4674-a97d-afc7deff0670 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:57.427086 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:57.465740 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:57.532558 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d98fd4-5b21-4196-aa96-762b087957be None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:57.533372 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28159588-508b-4674-a97d-afc7deff0670 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:57.572045 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec3312-943b-4387-9314-77aea046c2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:57.573106 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec3312-943b-4387-9314-77aea046c2a7 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:57.573280 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec3312-943b-4387-9314-77aea046c2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:57.573526 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec3312-943b-4387-9314-77aea046c2a7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:57.580043 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41049109-a04d-4365-ac38-d5810df6d426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:57.580722 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41049109-a04d-4365-ac38-d5810df6d426 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:57.581031 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41049109-a04d-4365-ac38-d5810df6d426 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:57.581305 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41049109-a04d-4365-ac38-d5810df6d426 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:57.582003 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41049109-a04d-4365-ac38-d5810df6d426 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:57.583396 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41049109-a04d-4365-ac38-d5810df6d426 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:57.585707 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdec3312-943b-4387-9314-77aea046c2a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:03:57.590913 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 491/977] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Jun 27 21:03:57 2022] GET /identity/v3/services => generated 1727 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:03:57.594710 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-cdec3312-943b-4387-9314-77aea046c2a7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 21:03:57.594710 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:57.594710 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:57.594710 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:57.594710 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:57.594710 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:57.594710 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:57.594710 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:57.594710 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:57.594710 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:57.594710 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:57.594710 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:57.594710 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:03:57.594710 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:03:57.594710 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:03:57.594710 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:03:57.595708 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:03:57.595708 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:03:57.595708 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:03:57.595708 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:03:57.595708 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:57.595708 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:57.595708 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:57.595708 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:57.595708 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:57.595708 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:57.595708 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:57.595708 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:57.595708 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:57.595708 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:57.595708 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:57.595708 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:57.596597 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:57.596597 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:57.596597 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:57.596597 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:57.596597 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:03:57.596597 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:03:57.596597 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:03:57.596597 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:03:57.596597 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:03:57.596597 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:03:57.596597 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 21:03:57.596597 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:57.602278 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 487/978] 10.222.0.13 () {56 vars in 1120 bytes} [Mon Jun 27 21:03:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:03:57.604346 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41049109-a04d-4365-ac38-d5810df6d426 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:57.737522 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41049109-a04d-4365-ac38-d5810df6d426 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:57.740160 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:57.785594 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26547966-b255-45e4-80b4-a566e0e0bc1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:57.786273 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26547966-b255-45e4-80b4-a566e0e0bc1a None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:57.786554 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26547966-b255-45e4-80b4-a566e0e0bc1a None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:57.786826 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26547966-b255-45e4-80b4-a566e0e0bc1a None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:57.787589 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26547966-b255-45e4-80b4-a566e0e0bc1a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:57.787896 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26547966-b255-45e4-80b4-a566e0e0bc1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:57.789301 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26547966-b255-45e4-80b4-a566e0e0bc1a None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:57.799161 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 492/979] 10.222.0.13 () {56 vars in 1115 bytes} [Mon Jun 27 21:03:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:57.805457 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec3312-943b-4387-9314-77aea046c2a7 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:57.884688 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:57.918972 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a67ca-b5d3-49fb-954d-7ad765d586d6 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:57.919303 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a67ca-b5d3-49fb-954d-7ad765d586d6 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:57.919518 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a67ca-b5d3-49fb-954d-7ad765d586d6 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:57.920904 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 493/980] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:03:57.926642 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7469b0-c972-4c94-b5c9-666221df8a80 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:57.927032 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7469b0-c972-4c94-b5c9-666221df8a80 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:57.927191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7469b0-c972-4c94-b5c9-666221df8a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:57.927953 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec3312-943b-4387-9314-77aea046c2a7 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:57.945755 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-9e7469b0-c972-4c94-b5c9-666221df8a80 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:57.945755 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-9e7469b0-c972-4c94-b5c9-666221df8a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:57.972702 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b74c1e8-5ee0-4f13-b2c5-7240291bdc72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:57.973395 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74c1e8-5ee0-4f13-b2c5-7240291bdc72 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:57.973673 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74c1e8-5ee0-4f13-b2c5-7240291bdc72 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:57.973948 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74c1e8-5ee0-4f13-b2c5-7240291bdc72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:57.985723 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b74c1e8-5ee0-4f13-b2c5-7240291bdc72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:57.987178 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b74c1e8-5ee0-4f13-b2c5-7240291bdc72 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:58.007637 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-4b74c1e8-5ee0-4f13-b2c5-7240291bdc72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:58.007637 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:58.007637 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:58.007637 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:58.007637 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:58.007637 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:58.007637 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:58.007637 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:58.007637 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:58.007637 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:58.007637 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:58.007637 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:58.007637 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:58.007637 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:58.007637 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:58.008714 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:58.008714 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:58.008714 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:58.008714 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:58.008714 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:58.008714 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:58.008714 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:58.008714 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:58.008714 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:58.008714 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:58.008714 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:58.008714 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:58.008714 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:58.008714 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:58.008714 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:58.008714 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:58.008714 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:58.013440 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:58.013440 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:58.013440 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:58.013440 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:58.013440 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 488/981] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:58.017055 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b74c1e8-5ee0-4f13-b2c5-7240291bdc72 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:58.045685 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 494/982] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:57 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:03:58.068962 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7469b0-c972-4c94-b5c9-666221df8a80 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:58.143695 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b74c1e8-5ee0-4f13-b2c5-7240291bdc72 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:58.166144 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:58.180106 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:58.188889 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7469b0-c972-4c94-b5c9-666221df8a80 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:58.189933 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab1796-ed8c-4d85-bebf-045678980f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:58.190678 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab1796-ed8c-4d85-bebf-045678980f99 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:58.191031 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab1796-ed8c-4d85-bebf-045678980f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:58.192047 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab1796-ed8c-4d85-bebf-045678980f99 None admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:58.192047 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab1796-ed8c-4d85-bebf-045678980f99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:58.192341 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab1796-ed8c-4d85-bebf-045678980f99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:58.193717 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab1796-ed8c-4d85-bebf-045678980f99 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:58.207150 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 489/983] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:58.212340 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab1796-ed8c-4d85-bebf-045678980f99 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:58.230728 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb06708-e440-41bf-acef-c1277f05a167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:58.231540 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb06708-e440-41bf-acef-c1277f05a167 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:58.231842 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb06708-e440-41bf-acef-c1277f05a167 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:58.232097 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb06708-e440-41bf-acef-c1277f05a167 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:58.232753 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb06708-e440-41bf-acef-c1277f05a167 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:58.234169 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb06708-e440-41bf-acef-c1277f05a167 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:58.246349 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-fcb06708-e440-41bf-acef-c1277f05a167 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 21:03:58.246349 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:58.246349 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:03:58.246349 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:03:58.246349 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:03:58.246349 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:03:58.246349 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:03:58.246349 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:58.246349 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 21:03:58.246349 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:58.246349 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:03:58.246349 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:58.246349 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:58.246349 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:58.246349 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:58.246349 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:58.246349 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:58.247373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:58.247373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:58.247373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:58.247373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:58.247373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:58.247373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:58.247373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:03:58.247373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:03:58.247373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:03:58.247373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:03:58.247373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:58.247373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:58.247373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:58.247373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:58.247373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:58.247373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:58.248229 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:58.248229 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:58.248229 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:58.248229 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:58.248229 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:58.248229 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:58.248229 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:58.248229 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:58.248229 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:58.248229 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:58.248229 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:03:58.248229 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:03:58.248229 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 21:03:58.248229 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:58.248229 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 495/984] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Jun 27 21:03:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:58.253458 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb06708-e440-41bf-acef-c1277f05a167 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:58.334435 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab1796-ed8c-4d85-bebf-045678980f99 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:58.366761 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb06708-e440-41bf-acef-c1277f05a167 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:58.379835 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d9825-932f-41f6-a390-79ff0780610a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:58.381377 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d9825-932f-41f6-a390-79ff0780610a None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:58.381706 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d9825-932f-41f6-a390-79ff0780610a None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:58.382044 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d9825-932f-41f6-a390-79ff0780610a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:58.392147 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197d9825-932f-41f6-a390-79ff0780610a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:58.394231 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197d9825-932f-41f6-a390-79ff0780610a None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:58.401549 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e08b73-36eb-4d91-a42b-fa1a83a28501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:58.402215 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e08b73-36eb-4d91-a42b-fa1a83a28501 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:58.402493 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e08b73-36eb-4d91-a42b-fa1a83a28501 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:58.402757 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e08b73-36eb-4d91-a42b-fa1a83a28501 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:58.403456 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e08b73-36eb-4d91-a42b-fa1a83a28501 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:58.403762 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e08b73-36eb-4d91-a42b-fa1a83a28501 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:58.405139 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e08b73-36eb-4d91-a42b-fa1a83a28501 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:58.409993 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 496/985] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Jun 27 21:03:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:58.413852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-197d9825-932f-41f6-a390-79ff0780610a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 21:03:58.413852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:58.413852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:58.413852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:58.413852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:58.413852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:58.413852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:58.413852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:58.413852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:58.413852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:58.413852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:58.413852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:58.413852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:03:58.413852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:03:58.413852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:03:58.413852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:03:58.414801 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:58.414801 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:58.414801 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:03:58.414801 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:58.414801 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:03:58.414801 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:03:58.414801 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:58.414801 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:58.414801 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:58.414801 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:58.414801 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:58.414801 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:58.414801 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:58.414801 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:58.414801 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:58.414801 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:58.414801 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:58.414801 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:58.415746 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:58.415746 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:58.415746 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:03:58.415746 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:03:58.415746 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:03:58.415746 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:03:58.415746 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:03:58.415746 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:03:58.415746 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 27 21:03:58.415746 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:03:58.421562 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e08b73-36eb-4d91-a42b-fa1a83a28501 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:58.421665 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 490/986] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Jun 27 21:03:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:58.421745 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d9825-932f-41f6-a390-79ff0780610a None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:58.444440 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:58.484840 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:58.530909 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e08b73-36eb-4d91-a42b-fa1a83a28501 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:58.548831 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d9825-932f-41f6-a390-79ff0780610a None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:58.568113 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26008fd0-d007-49da-b329-090ece2e936f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:58.568784 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26008fd0-d007-49da-b329-090ece2e936f None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:58.569100 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26008fd0-d007-49da-b329-090ece2e936f None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:58.569379 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26008fd0-d007-49da-b329-090ece2e936f None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:58.570092 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26008fd0-d007-49da-b329-090ece2e936f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:58.570403 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26008fd0-d007-49da-b329-090ece2e936f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:58.571794 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26008fd0-d007-49da-b329-090ece2e936f None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:58.577297 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 497/987] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Jun 27 21:03:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:58.583426 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26008fd0-d007-49da-b329-090ece2e936f None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:58.590956 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54a41e-0cbb-4c04-9d91-dd11fcccb7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:58.591873 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54a41e-0cbb-4c04-9d91-dd11fcccb7b8 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:58.592303 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54a41e-0cbb-4c04-9d91-dd11fcccb7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:58.592723 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54a41e-0cbb-4c04-9d91-dd11fcccb7b8 None admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:58.593762 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd54a41e-0cbb-4c04-9d91-dd11fcccb7b8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:58.594811 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd54a41e-0cbb-4c04-9d91-dd11fcccb7b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:58.596899 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd54a41e-0cbb-4c04-9d91-dd11fcccb7b8 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:58.627105 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 491/988] 10.222.0.13 () {56 vars in 1149 bytes} [Mon Jun 27 21:03:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:58.628522 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54a41e-0cbb-4c04-9d91-dd11fcccb7b8 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:58.684281 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:58.701135 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26008fd0-d007-49da-b329-090ece2e936f None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:58.740476 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:58.745684 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54a41e-0cbb-4c04-9d91-dd11fcccb7b8 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:58.748166 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d499c66-d555-416a-91c4-4dc3b48672fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:58.749589 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d499c66-d555-416a-91c4-4dc3b48672fb None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:58.750050 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d499c66-d555-416a-91c4-4dc3b48672fb None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:58.750509 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d499c66-d555-416a-91c4-4dc3b48672fb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:58.751905 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d499c66-d555-416a-91c4-4dc3b48672fb None admin] RBAC: Adding query filter params (interface=admin, service_id=06948e60ee834a0c94858c9a0052ede0, region_id=RegionOne) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:58.752437 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d499c66-d555-416a-91c4-4dc3b48672fb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:58.754855 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d499c66-d555-416a-91c4-4dc3b48672fb None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:58.762605 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 498/989] 10.222.0.13 () {56 vars in 1249 bytes} [Mon Jun 27 21:03:58 2022] GET /identity/v3/endpoints?service_id=06948e60ee834a0c94858c9a0052ede0&interface=admin®ion_id=RegionOne => generated 193 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:03:58.769045 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d499c66-d555-416a-91c4-4dc3b48672fb None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:58.787663 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8996e2-d47f-4e50-9171-080a69911d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:58.788308 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8996e2-d47f-4e50-9171-080a69911d1a None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:58.788680 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8996e2-d47f-4e50-9171-080a69911d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:58.789001 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8996e2-d47f-4e50-9171-080a69911d1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:58.863071 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8996e2-d47f-4e50-9171-080a69911d1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:58.864432 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8996e2-d47f-4e50-9171-080a69911d1a None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:58.881019 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d499c66-d555-416a-91c4-4dc3b48672fb None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:58.918190 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdb7d9e-f983-4a3a-b691-984d4708a1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:58.918862 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb7d9e-f983-4a3a-b691-984d4708a1df None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:58.919106 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb7d9e-f983-4a3a-b691-984d4708a1df None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:58.919378 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb7d9e-f983-4a3a-b691-984d4708a1df None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:58.920071 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdb7d9e-f983-4a3a-b691-984d4708a1df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:58.921509 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdb7d9e-f983-4a3a-b691-984d4708a1df None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:58.927860 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 499/990] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Jun 27 21:03:58 2022] GET /identity/v3/services => generated 1727 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:03:58.982942 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:58.993119 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 492/991] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:03:58 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 27 21:03:58.995369 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdb7d9e-f983-4a3a-b691-984d4708a1df None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:59.018881 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:59.124450 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdb7d9e-f983-4a3a-b691-984d4708a1df None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:59.179053 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fb00b0-ef40-4076-bdfb-bbeb6bc7fbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:59.180950 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb00b0-ef40-4076-bdfb-bbeb6bc7fbec None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:59.181744 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb00b0-ef40-4076-bdfb-bbeb6bc7fbec None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:59.182545 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb00b0-ef40-4076-bdfb-bbeb6bc7fbec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:59.195100 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fb00b0-ef40-4076-bdfb-bbeb6bc7fbec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:59.196976 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fb00b0-ef40-4076-bdfb-bbeb6bc7fbec None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:59.215903 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-41fb00b0-ef40-4076-bdfb-bbeb6bc7fbec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:59.215903 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:59.215903 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:59.215903 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:59.215903 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:59.215903 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:59.215903 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:59.215903 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:59.215903 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:59.215903 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:59.215903 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:59.215903 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:59.215903 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:03:59.215903 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:03:59.215903 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:03:59.216883 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:03:59.216883 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:59.216883 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:59.216883 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:59.216883 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:59.216883 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:59.216883 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:59.216883 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:59.216883 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:59.216883 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:59.216883 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:59.216883 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:59.216883 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:59.216883 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:59.216883 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:59.216883 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:59.216883 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:59.217855 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:03:59.217855 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:03:59.217855 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:03:59.217855 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:59.218922 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 500/992] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:03:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:59.225477 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8996e2-d47f-4e50-9171-080a69911d1a None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:59.393499 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8996e2-d47f-4e50-9171-080a69911d1a None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:59.451026 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82770416-3db5-491f-ab9c-d75a100974a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:59.451726 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82770416-3db5-491f-ab9c-d75a100974a3 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:59.452006 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82770416-3db5-491f-ab9c-d75a100974a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:59.452284 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82770416-3db5-491f-ab9c-d75a100974a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:59.453051 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82770416-3db5-491f-ab9c-d75a100974a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:59.453364 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82770416-3db5-491f-ab9c-d75a100974a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:59.454747 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82770416-3db5-491f-ab9c-d75a100974a3 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:59.467811 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 493/993] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:03:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:59.473308 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fb00b0-ef40-4076-bdfb-bbeb6bc7fbec None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:59.547586 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:59.591287 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fb00b0-ef40-4076-bdfb-bbeb6bc7fbec None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:59.628888 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ccbea0-0848-49cd-8949-aad3b280429e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:59.629702 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ccbea0-0848-49cd-8949-aad3b280429e None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:59.629987 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ccbea0-0848-49cd-8949-aad3b280429e None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:59.630261 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ccbea0-0848-49cd-8949-aad3b280429e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:59.637376 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ccbea0-0848-49cd-8949-aad3b280429e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:59.638842 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ccbea0-0848-49cd-8949-aad3b280429e None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:59.653475 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-84ccbea0-0848-49cd-8949-aad3b280429e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:59.653475 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:03:59.653475 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:03:59.653475 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:03:59.653475 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:03:59.653475 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:03:59.653475 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:03:59.653475 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:03:59.653475 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:03:59.653475 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:03:59.653475 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:03:59.653475 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:03:59.653475 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:03:59.653475 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:03:59.653475 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:03:59.655187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:03:59.655187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:03:59.655187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:03:59.655187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:03:59.655187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:03:59.655187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:03:59.655187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:03:59.655187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:03:59.655187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:03:59.655187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:03:59.655187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:03:59.655187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:03:59.655187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:03:59.655187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:03:59.655187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:03:59.655187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:03:59.655187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:03:59.657684 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:03:59.657684 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:03:59.657684 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:03:59.657684 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:03:59.657684 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:03:59.657684 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:03:59.657684 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:03:59.657684 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:03:59.661078 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 501/994] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:03:59.661280 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82770416-3db5-491f-ab9c-d75a100974a3 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:59.688588 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48bf397-a4cb-4de4-90d2-0def32ec36a2 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:59.688858 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48bf397-a4cb-4de4-90d2-0def32ec36a2 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:59.689184 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48bf397-a4cb-4de4-90d2-0def32ec36a2 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:59.693620 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 502/995] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:03:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:03:59.695548 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182e4ce-04dd-46ac-baca-6df846cdd80b None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:59.695828 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182e4ce-04dd-46ac-baca-6df846cdd80b None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:59.696098 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182e4ce-04dd-46ac-baca-6df846cdd80b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:59.714823 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-5182e4ce-04dd-46ac-baca-6df846cdd80b None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:03:59.716016 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-5182e4ce-04dd-46ac-baca-6df846cdd80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:03:59.778491 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:59.806602 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 503/996] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:03:59 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:03:59.833971 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82770416-3db5-491f-ab9c-d75a100974a3 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:03:59.835910 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5182e4ce-04dd-46ac-baca-6df846cdd80b None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:59.873879 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa815a4-0057-4aa3-b9b5-8d327bbb372e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:03:59.874768 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa815a4-0057-4aa3-b9b5-8d327bbb372e None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:03:59.875049 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa815a4-0057-4aa3-b9b5-8d327bbb372e None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:03:59.875320 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa815a4-0057-4aa3-b9b5-8d327bbb372e None admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:03:59.876121 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa815a4-0057-4aa3-b9b5-8d327bbb372e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:03:59.876427 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa815a4-0057-4aa3-b9b5-8d327bbb372e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:03:59.877915 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa815a4-0057-4aa3-b9b5-8d327bbb372e None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:03:59.899868 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 494/997] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 27 21:03:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:03:59.901342 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa815a4-0057-4aa3-b9b5-8d327bbb372e None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:03:59.914664 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:03:59.968820 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5182e4ce-04dd-46ac-baca-6df846cdd80b None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:00.033014 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744143d4-96f0-470d-9d50-4acff7c2ddc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:00.033919 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744143d4-96f0-470d-9d50-4acff7c2ddc1 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:00.034271 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744143d4-96f0-470d-9d50-4acff7c2ddc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:00.034625 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744143d4-96f0-470d-9d50-4acff7c2ddc1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:00.035367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744143d4-96f0-470d-9d50-4acff7c2ddc1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:00.036823 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744143d4-96f0-470d-9d50-4acff7c2ddc1 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:00.047385 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa815a4-0057-4aa3-b9b5-8d327bbb372e None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:00.049177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-744143d4-96f0-470d-9d50-4acff7c2ddc1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 21:04:00.049177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:00.049177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:04:00.049177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:04:00.049177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:04:00.049177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:04:00.049177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:04:00.049177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:00.049177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 21:04:00.049177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:04:00.049177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:04:00.049177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:04:00.049177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:00.049177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:00.049177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:00.049177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:00.049177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:00.050172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:00.050172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:00.050172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:00.050172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:00.050172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:00.050172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:00.050172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:04:00.050172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:04:00.050172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:04:00.050172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:04:00.050172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:00.050172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:00.050172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:00.050172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:00.050172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:00.050172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:00.051046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:00.051046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:00.051046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:00.051046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:00.051046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:00.051046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:00.051046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:00.051046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:00.051046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:00.051046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:00.051046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:04:00.051046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:00.051046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 27 21:04:00.051046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:04:00.053399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 504/998] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:03:59 2022] GET /identity/v3/services/compute => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:04:00.056579 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744143d4-96f0-470d-9d50-4acff7c2ddc1 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:00.082379 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:00.104439 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb873de-c7f4-4fdc-9ffe-1c6703ef888b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:00.105155 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb873de-c7f4-4fdc-9ffe-1c6703ef888b None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:00.105555 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb873de-c7f4-4fdc-9ffe-1c6703ef888b None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:00.105756 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb873de-c7f4-4fdc-9ffe-1c6703ef888b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:00.106618 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb873de-c7f4-4fdc-9ffe-1c6703ef888b None admin] RBAC: Adding query filter params (role.id=b3dcc5de8383479d995a9bc38d8a8ef2, scope.project.id=334438c929c44e489e9a3b93e6d37231, user.id=f67110d591a94537a3df6a2d835370f9) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:00.107751 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb873de-c7f4-4fdc-9ffe-1c6703ef888b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:00.108398 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb873de-c7f4-4fdc-9ffe-1c6703ef888b None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:00.116125 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 495/999] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:03:59 2022] GET /identity/v3/role_assignments?user.id=f67110d591a94537a3df6a2d835370f9&scope.project.id=334438c929c44e489e9a3b93e6d37231&role.id=b3dcc5de8383479d995a9bc38d8a8ef2 => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:04:00.204005 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744143d4-96f0-470d-9d50-4acff7c2ddc1 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:00.237781 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:00.254612 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c08d67-3e49-4cee-bcc5-599aaed54c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:00.255463 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c08d67-3e49-4cee-bcc5-599aaed54c3d None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:00.255842 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c08d67-3e49-4cee-bcc5-599aaed54c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:00.256236 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c08d67-3e49-4cee-bcc5-599aaed54c3d None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:00.257112 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c08d67-3e49-4cee-bcc5-599aaed54c3d None admin] RBAC: Adding query filter params (name=compute) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:00.257523 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c08d67-3e49-4cee-bcc5-599aaed54c3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:00.259030 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c08d67-3e49-4cee-bcc5-599aaed54c3d None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:00.265294 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 505/1000] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Jun 27 21:04:00 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:04:00.271159 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb873de-c7f4-4fdc-9ffe-1c6703ef888b None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:00.389712 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb873de-c7f4-4fdc-9ffe-1c6703ef888b None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:00.425271 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b5ce1-bb43-4131-bb8c-683e64ff3b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:00.426302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b5ce1-bb43-4131-bb8c-683e64ff3b39 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:00.426769 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b5ce1-bb43-4131-bb8c-683e64ff3b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:00.427243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b5ce1-bb43-4131-bb8c-683e64ff3b39 None admin] PATH_INFO: `/v3/services` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:00.428143 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817b5ce1-bb43-4131-bb8c-683e64ff3b39 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:00.428627 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817b5ce1-bb43-4131-bb8c-683e64ff3b39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:00.430212 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817b5ce1-bb43-4131-bb8c-683e64ff3b39 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:00.436514 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 496/1001] 10.222.0.13 () {56 vars in 1112 bytes} [Mon Jun 27 21:04:00 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:00.442236 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c08d67-3e49-4cee-bcc5-599aaed54c3d None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:00.549375 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c08d67-3e49-4cee-bcc5-599aaed54c3d None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:00.583657 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88e265c-41c6-4e59-8328-138abc40b24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:00.584322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e265c-41c6-4e59-8328-138abc40b24f None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:00.584604 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e265c-41c6-4e59-8328-138abc40b24f None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:00.584877 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e265c-41c6-4e59-8328-138abc40b24f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:00.585662 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88e265c-41c6-4e59-8328-138abc40b24f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:00.587042 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88e265c-41c6-4e59-8328-138abc40b24f None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:00.616966 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 506/1002] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Jun 27 21:04:00 2022] POST /identity/v3/endpoints => generated 334 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:04:01.045817 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca8a07-4679-4a01-8ff6-a37a154c9548 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:01.046494 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca8a07-4679-4a01-8ff6-a37a154c9548 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:01.046494 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca8a07-4679-4a01-8ff6-a37a154c9548 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:01.047783 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 497/1003] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:04:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:01.053439 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25349c50-a390-4ee0-8edf-cc64dcfc7888 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:01.053703 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25349c50-a390-4ee0-8edf-cc64dcfc7888 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:01.053981 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25349c50-a390-4ee0-8edf-cc64dcfc7888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:01.082885 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-25349c50-a390-4ee0-8edf-cc64dcfc7888 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:01.082885 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-25349c50-a390-4ee0-8edf-cc64dcfc7888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:01.123938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:01.167918 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 507/1004] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:01 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:01.192010 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca8a07-4679-4a01-8ff6-a37a154c9548 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:01.246533 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:01.310118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca8a07-4679-4a01-8ff6-a37a154c9548 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:01.346720 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b8440f-3832-455b-85db-3be2deb4ef64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:01.347546 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8440f-3832-455b-85db-3be2deb4ef64 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:01.347826 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8440f-3832-455b-85db-3be2deb4ef64 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:01.348104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8440f-3832-455b-85db-3be2deb4ef64 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:01.348766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b8440f-3832-455b-85db-3be2deb4ef64 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:01.350169 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b8440f-3832-455b-85db-3be2deb4ef64 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:01.363443 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-80b8440f-3832-455b-85db-3be2deb4ef64 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 21:04:01.363443 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:01.363443 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:04:01.363443 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:04:01.363443 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:04:01.363443 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:04:01.363443 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:04:01.363443 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:01.363443 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 21:04:01.363443 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:04:01.363443 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:04:01.363443 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:04:01.363443 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:01.363443 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:01.363443 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:01.363443 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:01.363443 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:01.364527 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:01.364527 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:01.364527 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:01.364527 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:01.364527 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:01.364527 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:01.364527 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:04:01.364527 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:04:01.364527 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:04:01.364527 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:04:01.364527 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:01.364527 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:01.364527 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:01.364527 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:01.364527 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:01.364527 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:01.365451 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:01.365451 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:01.365451 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:01.365451 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:01.365451 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:01.365451 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:01.365451 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:01.365451 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:01.365451 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:01.365451 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:01.365451 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:04:01.365451 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:01.365451 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 27 21:04:01.365451 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:04:01.365451 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 498/1005] 10.222.0.13 () {56 vars in 1126 bytes} [Mon Jun 27 21:04:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:01.370989 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25349c50-a390-4ee0-8edf-cc64dcfc7888 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:01.475444 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25349c50-a390-4ee0-8edf-cc64dcfc7888 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:01.517863 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f109052-5371-40d8-bffa-eea055d0c9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:01.518529 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f109052-5371-40d8-bffa-eea055d0c9d2 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:01.518987 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f109052-5371-40d8-bffa-eea055d0c9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:01.519261 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f109052-5371-40d8-bffa-eea055d0c9d2 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:01.519965 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f109052-5371-40d8-bffa-eea055d0c9d2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:01.520279 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f109052-5371-40d8-bffa-eea055d0c9d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:01.521675 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f109052-5371-40d8-bffa-eea055d0c9d2 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:01.527374 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 508/1006] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Jun 27 21:04:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:01.533064 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b8440f-3832-455b-85db-3be2deb4ef64 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:01.639839 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b8440f-3832-455b-85db-3be2deb4ef64 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:01.650673 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:01.676473 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f167fb47-a31e-4051-9e13-d9e814e1fd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:01.677474 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f167fb47-a31e-4051-9e13-d9e814e1fd50 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:01.677903 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f167fb47-a31e-4051-9e13-d9e814e1fd50 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:01.678322 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f167fb47-a31e-4051-9e13-d9e814e1fd50 None admin] PATH_INFO: `/v3/services` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:01.679222 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f167fb47-a31e-4051-9e13-d9e814e1fd50 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:01.679673 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f167fb47-a31e-4051-9e13-d9e814e1fd50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:01.681240 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f167fb47-a31e-4051-9e13-d9e814e1fd50 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:01.691816 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 499/1007] 10.222.0.13 () {56 vars in 1122 bytes} [Mon Jun 27 21:04:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:01.693303 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f109052-5371-40d8-bffa-eea055d0c9d2 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:01.735388 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:01.825370 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f109052-5371-40d8-bffa-eea055d0c9d2 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:01.868981 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d1d99-846b-44ea-a076-0ee69c7458d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:01.870002 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d1d99-846b-44ea-a076-0ee69c7458d3 None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:01.870470 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d1d99-846b-44ea-a076-0ee69c7458d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:01.870960 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d1d99-846b-44ea-a076-0ee69c7458d3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:01.871881 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d1d99-846b-44ea-a076-0ee69c7458d3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:01.873504 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d1d99-846b-44ea-a076-0ee69c7458d3 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:01.897109 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 509/1008] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Jun 27 21:04:01 2022] POST /identity/v3/endpoints => generated 325 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:04:02.259229 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522bfb8b-2f07-4923-860d-7202a46e1ed1 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:02.259827 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522bfb8b-2f07-4923-860d-7202a46e1ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:02.259827 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522bfb8b-2f07-4923-860d-7202a46e1ed1 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:02.261548 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 500/1009] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:04:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:02.267066 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9bf76b-99b6-472c-91b8-dedf6b259aef None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:02.267367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9bf76b-99b6-472c-91b8-dedf6b259aef None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:02.267633 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9bf76b-99b6-472c-91b8-dedf6b259aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:02.286441 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-8c9bf76b-99b6-472c-91b8-dedf6b259aef None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:02.287654 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-8c9bf76b-99b6-472c-91b8-dedf6b259aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:02.370555 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 510/1010] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:02 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:02.393823 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522bfb8b-2f07-4923-860d-7202a46e1ed1 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:02.515964 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522bfb8b-2f07-4923-860d-7202a46e1ed1 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:02.555256 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d1dd7-9acb-4db3-80b9-7c2692567148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:02.556057 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d1dd7-9acb-4db3-80b9-7c2692567148 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:02.556339 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d1dd7-9acb-4db3-80b9-7c2692567148 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:02.556587 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d1dd7-9acb-4db3-80b9-7c2692567148 None admin] PATH_INFO: `/v3/services/image` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:02.557334 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73d1dd7-9acb-4db3-80b9-7c2692567148 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:02.558815 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73d1dd7-9acb-4db3-80b9-7c2692567148 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:02.571472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-f73d1dd7-9acb-4db3-80b9-7c2692567148 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 21:04:02.571472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:02.571472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:04:02.571472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:04:02.571472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:04:02.571472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:04:02.571472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:04:02.571472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:02.571472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 21:04:02.571472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:04:02.571472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:04:02.571472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:04:02.571472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:02.571472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:02.571472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:02.571472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:02.571472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:02.578148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:02.578148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:02.578148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:02.578148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:02.578148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:02.578148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:02.578148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:04:02.578148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:04:02.578148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:04:02.578148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:04:02.578148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:02.578148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:02.578148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:02.578148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:02.578148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:02.578148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:02.579032 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:02.579032 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:02.579032 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:02.579032 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:02.579032 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:02.579032 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:02.579032 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:02.579032 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:02.579032 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:02.579032 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:02.579032 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:04:02.579032 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:02.579032 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 21:04:02.579032 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:04:02.579032 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 501/1011] 10.222.0.13 () {56 vars in 1105 bytes} [Mon Jun 27 21:04:02 2022] GET /identity/v3/services/image => generated 86 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:02.579869 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9bf76b-99b6-472c-91b8-dedf6b259aef None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:02.600562 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:02.704058 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9bf76b-99b6-472c-91b8-dedf6b259aef None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:02.744293 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d360c9-9e33-4f80-baea-5de4d3c48050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:02.745585 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d360c9-9e33-4f80-baea-5de4d3c48050 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:02.745863 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d360c9-9e33-4f80-baea-5de4d3c48050 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:02.746135 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d360c9-9e33-4f80-baea-5de4d3c48050 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:02.746838 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d360c9-9e33-4f80-baea-5de4d3c48050 None admin] RBAC: Adding query filter params (name=image) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:02.747140 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d360c9-9e33-4f80-baea-5de4d3c48050 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:02.748503 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d360c9-9e33-4f80-baea-5de4d3c48050 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:02.754445 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 511/1012] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:02.760121 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d360c9-9e33-4f80-baea-5de4d3c48050 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:02.760496 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef1a54-f4e8-4dde-a3a1-4a0e5b1918b9 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:02.760789 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef1a54-f4e8-4dde-a3a1-4a0e5b1918b9 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:02.761104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef1a54-f4e8-4dde-a3a1-4a0e5b1918b9 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:02.762672 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 502/1013] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:04:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:02.767965 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584dfacf-5b19-4285-9367-73221cc3446d None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:02.768208 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584dfacf-5b19-4285-9367-73221cc3446d None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:02.768440 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584dfacf-5b19-4285-9367-73221cc3446d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:02.788401 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-584dfacf-5b19-4285-9367-73221cc3446d None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:02.789626 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-584dfacf-5b19-4285-9367-73221cc3446d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:02.833084 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:02.864497 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:02.877798 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 503/1014] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:02 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:04:02.896017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d360c9-9e33-4f80-baea-5de4d3c48050 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:02.901862 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584dfacf-5b19-4285-9367-73221cc3446d None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:02.937318 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf8a56-68a6-4540-a30e-85d4ddd12632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:02.937992 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf8a56-68a6-4540-a30e-85d4ddd12632 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:02.938275 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf8a56-68a6-4540-a30e-85d4ddd12632 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:02.938546 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf8a56-68a6-4540-a30e-85d4ddd12632 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:02.939262 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaf8a56-68a6-4540-a30e-85d4ddd12632 None admin] RBAC: Adding query filter params (type=image) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:02.939571 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaf8a56-68a6-4540-a30e-85d4ddd12632 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:02.940979 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaf8a56-68a6-4540-a30e-85d4ddd12632 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:02.952964 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 512/1015] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:02 2022] GET /identity/v3/services?type=image => generated 122 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:04:03.044207 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584dfacf-5b19-4285-9367-73221cc3446d None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:03.089249 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c20756-bcb0-4b7b-ac8e-1a98717cf401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:03.090096 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c20756-bcb0-4b7b-ac8e-1a98717cf401 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:03.090368 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c20756-bcb0-4b7b-ac8e-1a98717cf401 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:03.090645 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c20756-bcb0-4b7b-ac8e-1a98717cf401 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:03.105310 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c20756-bcb0-4b7b-ac8e-1a98717cf401 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:04:03.114659 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-49c20756-bcb0-4b7b-ac8e-1a98717cf401 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 21:04:03.114659 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:03.114659 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:03.114659 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:03.114659 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:03.114659 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:03.114659 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:03.114659 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:03.114659 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:03.114659 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:03.114659 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:03.114659 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:03.114659 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:04:03.114659 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:04:03.114659 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:04:03.114659 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:04:03.115681 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:04:03.115681 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:04:03.115681 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:04:03.115681 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:04:03.115681 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:03.115681 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:03.115681 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:03.115681 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:03.115681 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:03.115681 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:03.115681 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:03.115681 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:03.115681 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:03.115681 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:03.115681 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:03.115681 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:03.116648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:03.116648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:03.116648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:03.116648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:03.116648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:04:03.116648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:04:03.116648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:04:03.116648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:04:03.116648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:04:03.116648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:04:03.116648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 21:04:03.116648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:04:03.116648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 504/1016] 10.222.0.13 () {56 vars in 1120 bytes} [Mon Jun 27 21:04:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:03.122820 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf8a56-68a6-4540-a30e-85d4ddd12632 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:03.244425 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf8a56-68a6-4540-a30e-85d4ddd12632 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:03.285096 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b8c9f2-1e39-4e47-917e-83bd6b418a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:03.285096 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b8c9f2-1e39-4e47-917e-83bd6b418a05 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:03.285096 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b8c9f2-1e39-4e47-917e-83bd6b418a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:03.285096 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b8c9f2-1e39-4e47-917e-83bd6b418a05 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:03.285495 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b8c9f2-1e39-4e47-917e-83bd6b418a05 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:03.285495 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b8c9f2-1e39-4e47-917e-83bd6b418a05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:03.286584 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b8c9f2-1e39-4e47-917e-83bd6b418a05 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:03.296112 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 513/1017] 10.222.0.13 () {56 vars in 1115 bytes} [Mon Jun 27 21:04:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:03.301850 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c20756-bcb0-4b7b-ac8e-1a98717cf401 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:03.345460 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:03.437324 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c20756-bcb0-4b7b-ac8e-1a98717cf401 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:03.479597 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7696ccf-7f87-416e-9e70-9bb3cfbe5bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:03.480429 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7696ccf-7f87-416e-9e70-9bb3cfbe5bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:03.480711 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7696ccf-7f87-416e-9e70-9bb3cfbe5bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:03.481014 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7696ccf-7f87-416e-9e70-9bb3cfbe5bd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:03.493915 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7696ccf-7f87-416e-9e70-9bb3cfbe5bd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:03.495668 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7696ccf-7f87-416e-9e70-9bb3cfbe5bd2 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:03.514378 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-c7696ccf-7f87-416e-9e70-9bb3cfbe5bd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:03.514378 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:03.514378 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:03.514378 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:03.514378 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:03.514378 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:03.514378 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:03.514378 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:03.514378 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:03.514378 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:03.514378 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:03.514378 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:03.514378 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:04:03.514378 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:04:03.514378 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:04:03.515512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:04:03.515512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:03.515512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:03.515512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:03.515512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:03.515512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:03.515512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:03.515512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:03.515512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:03.515512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:03.515512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:03.515512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:03.515512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:03.515512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:03.515512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:03.515512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:03.515512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:03.521628 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:04:03.521628 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:04:03.521628 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:03.521628 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:04:03.521628 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 505/1018] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:03.521940 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b8c9f2-1e39-4e47-917e-83bd6b418a05 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:03.532119 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:03.628679 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b8c9f2-1e39-4e47-917e-83bd6b418a05 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:03.663616 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e858eb97-b44c-4c31-a3c7-1bdbd14e9809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:03.664272 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858eb97-b44c-4c31-a3c7-1bdbd14e9809 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:03.664568 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858eb97-b44c-4c31-a3c7-1bdbd14e9809 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:03.664840 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858eb97-b44c-4c31-a3c7-1bdbd14e9809 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:03.665607 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e858eb97-b44c-4c31-a3c7-1bdbd14e9809 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:03.665913 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e858eb97-b44c-4c31-a3c7-1bdbd14e9809 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:03.667334 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e858eb97-b44c-4c31-a3c7-1bdbd14e9809 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:03.680777 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 514/1019] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:04:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:03.686399 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7696ccf-7f87-416e-9e70-9bb3cfbe5bd2 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:03.794914 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:03.820271 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7696ccf-7f87-416e-9e70-9bb3cfbe5bd2 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:03.875421 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134dfdd8-70d1-4017-ad38-83760cdaa641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:03.876260 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134dfdd8-70d1-4017-ad38-83760cdaa641 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:03.876520 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134dfdd8-70d1-4017-ad38-83760cdaa641 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:03.876798 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134dfdd8-70d1-4017-ad38-83760cdaa641 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:03.885723 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134dfdd8-70d1-4017-ad38-83760cdaa641 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:03.887110 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134dfdd8-70d1-4017-ad38-83760cdaa641 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:03.901859 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-134dfdd8-70d1-4017-ad38-83760cdaa641 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 21:04:03.901859 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:03.901859 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:03.901859 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:03.901859 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:03.901859 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:03.901859 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:03.901859 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:03.901859 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:03.901859 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:03.901859 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:03.901859 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:03.901859 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:04:03.901859 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:04:03.901859 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:04:03.901859 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:04:03.903014 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:03.903014 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:03.903014 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:04:03.903014 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:04:03.903014 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:04:03.903014 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:04:03.903014 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:03.903014 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:03.903014 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:03.903014 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:03.903014 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:03.903014 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:03.903014 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:03.903014 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:03.903014 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:03.903014 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:03.903014 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:03.903014 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:03.908734 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:03.908734 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:03.908734 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:04:03.908734 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:04:03.908734 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:04:03.908734 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:04:03.908734 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:04:03.908734 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:04:03.908734 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 21:04:03.908734 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:04:03.908734 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 506/1020] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Jun 27 21:04:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:03.909391 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e858eb97-b44c-4c31-a3c7-1bdbd14e9809 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:03.986246 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebcc69-e6f4-4621-b996-1d17586b334b None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:03.986592 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebcc69-e6f4-4621-b996-1d17586b334b None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:03.986786 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebcc69-e6f4-4621-b996-1d17586b334b None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:03.987647 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:03.988433 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 507/1021] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:04:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:03.994647 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165eb0b2-606f-4474-8cf2-a5fe382b909e None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:03.995017 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165eb0b2-606f-4474-8cf2-a5fe382b909e None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:03.996161 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165eb0b2-606f-4474-8cf2-a5fe382b909e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:04.016610 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-165eb0b2-606f-4474-8cf2-a5fe382b909e None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:04.017879 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-165eb0b2-606f-4474-8cf2-a5fe382b909e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:04.039133 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e858eb97-b44c-4c31-a3c7-1bdbd14e9809 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:04.083949 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c55afb2-0ff9-4154-b017-f29839e2308c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:04.084641 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55afb2-0ff9-4154-b017-f29839e2308c None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:04.085035 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55afb2-0ff9-4154-b017-f29839e2308c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:04.086075 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55afb2-0ff9-4154-b017-f29839e2308c None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:04.086075 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c55afb2-0ff9-4154-b017-f29839e2308c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:04.086452 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c55afb2-0ff9-4154-b017-f29839e2308c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:04.087842 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c55afb2-0ff9-4154-b017-f29839e2308c None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:04.099579 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 508/1022] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:03 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:04:04.112806 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 515/1023] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Jun 27 21:04:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:04.118220 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165eb0b2-606f-4474-8cf2-a5fe382b909e None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:04.124367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c55afb2-0ff9-4154-b017-f29839e2308c None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:04.211936 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:04.253770 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c55afb2-0ff9-4154-b017-f29839e2308c None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:04.258032 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165eb0b2-606f-4474-8cf2-a5fe382b909e None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:04.280656 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:04.295041 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff90114-4b10-4b84-8aa2-d8f029e2168b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:04.295786 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff90114-4b10-4b84-8aa2-d8f029e2168b None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:04.296060 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff90114-4b10-4b84-8aa2-d8f029e2168b None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:04.296340 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff90114-4b10-4b84-8aa2-d8f029e2168b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:04.297335 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861888c2-f163-4bd7-b48e-2ec0c7e99c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:04.297574 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff90114-4b10-4b84-8aa2-d8f029e2168b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:04.298011 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861888c2-f163-4bd7-b48e-2ec0c7e99c8e None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:04.298289 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861888c2-f163-4bd7-b48e-2ec0c7e99c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:04.298563 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861888c2-f163-4bd7-b48e-2ec0c7e99c8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:04.298967 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff90114-4b10-4b84-8aa2-d8f029e2168b None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:04.305185 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 516/1024] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:04.307508 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff90114-4b10-4b84-8aa2-d8f029e2168b None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:04.389798 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861888c2-f163-4bd7-b48e-2ec0c7e99c8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:04.391356 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861888c2-f163-4bd7-b48e-2ec0c7e99c8e None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:04.427858 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff90114-4b10-4b84-8aa2-d8f029e2168b None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:04.467858 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ad47b5-b51c-47cc-af23-08b235a7949a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:04.468444 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad47b5-b51c-47cc-af23-08b235a7949a None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:04.468726 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad47b5-b51c-47cc-af23-08b235a7949a None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:04.469011 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad47b5-b51c-47cc-af23-08b235a7949a None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:04.469643 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:04.470546 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ad47b5-b51c-47cc-af23-08b235a7949a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:04.472061 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ad47b5-b51c-47cc-af23-08b235a7949a None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:04.509092 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 509/1025] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:04:04 2022] GET /identity/v3/auth/tokens => generated 1863 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 21:04:04.510426 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 517/1026] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Jun 27 21:04:04 2022] POST /identity/v3/projects => generated 311 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:04:04.518399 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861888c2-f163-4bd7-b48e-2ec0c7e99c8e None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:04.651791 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861888c2-f163-4bd7-b48e-2ec0c7e99c8e None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:04.697090 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c593be9e-c8e8-4ddd-a65b-5a04914ffad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:04.698133 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593be9e-c8e8-4ddd-a65b-5a04914ffad4 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:04.698372 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593be9e-c8e8-4ddd-a65b-5a04914ffad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:04.698639 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593be9e-c8e8-4ddd-a65b-5a04914ffad4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:04.715655 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c593be9e-c8e8-4ddd-a65b-5a04914ffad4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:04.717193 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c593be9e-c8e8-4ddd-a65b-5a04914ffad4 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:04.741194 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-c593be9e-c8e8-4ddd-a65b-5a04914ffad4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:04.741194 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:04.741194 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:04.741194 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:04.741194 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:04.741194 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:04.741194 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:04.741194 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:04.741194 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:04.741194 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:04.741194 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:04.741194 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:04.741194 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:04:04.741194 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:04:04.741194 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:04:04.742376 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:04:04.742376 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:04.742376 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:04.742376 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:04.742376 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:04.742376 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:04.742376 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:04.742376 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:04.742376 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:04.742376 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:04.742376 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:04.742376 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:04.742376 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:04.742376 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:04.742376 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:04.742376 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:04.742376 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:04.743464 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:04:04.743464 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:04:04.743464 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:04.743464 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:04:04.743464 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 510/1027] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:04:04.748891 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ad47b5-b51c-47cc-af23-08b235a7949a None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:04.803862 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:04.867376 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ad47b5-b51c-47cc-af23-08b235a7949a None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:04.905253 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd200fb-331c-46d2-a117-230f7baee16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:04.905917 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd200fb-331c-46d2-a117-230f7baee16f None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:04.906199 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd200fb-331c-46d2-a117-230f7baee16f None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:04.906476 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd200fb-331c-46d2-a117-230f7baee16f None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:04.907200 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd200fb-331c-46d2-a117-230f7baee16f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:04.907526 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd200fb-331c-46d2-a117-230f7baee16f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:04.908908 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd200fb-331c-46d2-a117-230f7baee16f None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:04.926472 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 518/1028] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:04:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:04.931960 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c593be9e-c8e8-4ddd-a65b-5a04914ffad4 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:04.961073 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:05.035000 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31621338-9229-4f17-a8fe-b31b4700235b None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:05.035242 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31621338-9229-4f17-a8fe-b31b4700235b None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:05.035522 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31621338-9229-4f17-a8fe-b31b4700235b None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:05.036910 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 519/1029] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:04:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:05.042425 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454982e-3d05-47d6-a971-fcf701cc55bd None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:05.042673 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454982e-3d05-47d6-a971-fcf701cc55bd None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:05.042979 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454982e-3d05-47d6-a971-fcf701cc55bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:05.055848 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c593be9e-c8e8-4ddd-a65b-5a04914ffad4 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:05.061973 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-6454982e-3d05-47d6-a971-fcf701cc55bd None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:05.063172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-6454982e-3d05-47d6-a971-fcf701cc55bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:05.094765 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a5d750-d73e-43c0-847a-901ee6dfe4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:05.095577 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5d750-d73e-43c0-847a-901ee6dfe4db None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:05.095861 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5d750-d73e-43c0-847a-901ee6dfe4db None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:05.096143 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5d750-d73e-43c0-847a-901ee6dfe4db None admin] PATH_INFO: `/v3/projects/service` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:05.104309 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a5d750-d73e-43c0-847a-901ee6dfe4db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:05.105822 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a5d750-d73e-43c0-847a-901ee6dfe4db None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:05.120733 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-e1a5d750-d73e-43c0-847a-901ee6dfe4db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:04:05.120733 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:05.120733 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:05.120733 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:05.120733 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:05.120733 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:05.120733 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:05.120733 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:05.120733 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:05.120733 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:05.120733 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:05.120733 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:05.120733 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:04:05.120733 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:04:05.120733 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:04:05.121852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:04:05.121852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:05.121852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:05.121852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:05.121852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:05.121852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:05.121852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:05.121852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:05.121852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:05.121852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:05.121852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:05.121852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:05.121852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:05.121852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:05.121852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:05.121852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:05.121852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:05.122913 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:04:05.122913 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:04:05.122913 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:04:05.122913 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:04:05.122913 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:04:05.122913 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:04:05.122913 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:04:05.122913 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:04:05.126892 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 511/1030] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:04:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:05.127940 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a5d750-d73e-43c0-847a-901ee6dfe4db None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:05.147651 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 520/1031] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:05 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:04:05.170944 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454982e-3d05-47d6-a971-fcf701cc55bd None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:05.209369 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:05.235737 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:05.257182 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a5d750-d73e-43c0-847a-901ee6dfe4db None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:05.288011 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454982e-3d05-47d6-a971-fcf701cc55bd None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:05.299336 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e8d0ad-21d0-4ae1-8047-3d7021e5eb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:05.300188 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8d0ad-21d0-4ae1-8047-3d7021e5eb53 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:05.300905 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8d0ad-21d0-4ae1-8047-3d7021e5eb53 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:05.301505 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8d0ad-21d0-4ae1-8047-3d7021e5eb53 None admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:05.302518 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e8d0ad-21d0-4ae1-8047-3d7021e5eb53 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:05.302991 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e8d0ad-21d0-4ae1-8047-3d7021e5eb53 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:05.305369 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e8d0ad-21d0-4ae1-8047-3d7021e5eb53 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:05.323158 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 512/1032] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 27 21:04:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:04:05.326981 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d4bcaf-a5ba-4174-9904-33ce703e2bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:05.327896 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4bcaf-a5ba-4174-9904-33ce703e2bce None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:05.328174 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4bcaf-a5ba-4174-9904-33ce703e2bce None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:05.328445 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d4bcaf-a5ba-4174-9904-33ce703e2bce None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:05.328900 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e8d0ad-21d0-4ae1-8047-3d7021e5eb53 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:05.329234 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d4bcaf-a5ba-4174-9904-33ce703e2bce None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:05.330618 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d4bcaf-a5ba-4174-9904-33ce703e2bce None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:05.353004 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 521/1033] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Jun 27 21:04:05 2022] POST /identity/v3/services => generated 242 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:04:05.466921 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e8d0ad-21d0-4ae1-8047-3d7021e5eb53 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:05.516050 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c13c09-3ef9-4b9a-8adf-094ad6f6a51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:05.516727 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c13c09-3ef9-4b9a-8adf-094ad6f6a51e None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:05.517662 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c13c09-3ef9-4b9a-8adf-094ad6f6a51e None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:05.517662 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c13c09-3ef9-4b9a-8adf-094ad6f6a51e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:05.518167 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c13c09-3ef9-4b9a-8adf-094ad6f6a51e None admin] RBAC: Adding query filter params (role.id=b3dcc5de8383479d995a9bc38d8a8ef2, scope.project.id=334438c929c44e489e9a3b93e6d37231, user.id=1366958034d540b4ba2feb421a6afabc) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:05.518481 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c13c09-3ef9-4b9a-8adf-094ad6f6a51e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:05.519943 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c13c09-3ef9-4b9a-8adf-094ad6f6a51e None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:05.526971 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 513/1034] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:04:05 2022] GET /identity/v3/role_assignments?user.id=1366958034d540b4ba2feb421a6afabc&scope.project.id=334438c929c44e489e9a3b93e6d37231&role.id=b3dcc5de8383479d995a9bc38d8a8ef2 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:06.603978 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac3f97-02e3-4a46-843a-fb73e2d298ce None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:06.604603 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac3f97-02e3-4a46-843a-fb73e2d298ce None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:06.604603 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac3f97-02e3-4a46-843a-fb73e2d298ce None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:06.605926 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 522/1035] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:04:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:06.612041 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be9242-a44b-47a7-9492-87e798833732 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:06.613285 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be9242-a44b-47a7-9492-87e798833732 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:06.613285 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be9242-a44b-47a7-9492-87e798833732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:06.618453 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:06.632787 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-09be9242-a44b-47a7-9492-87e798833732 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:06.634044 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-09be9242-a44b-47a7-9492-87e798833732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:06.717940 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 514/1036] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:06.742979 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac3f97-02e3-4a46-843a-fb73e2d298ce None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:06.853241 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:06.866887 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac3f97-02e3-4a46-843a-fb73e2d298ce None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:06.904260 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f5a768-8fa8-437d-ad2a-c2cb1233ed7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:06.905100 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5a768-8fa8-437d-ad2a-c2cb1233ed7f None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:06.905383 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5a768-8fa8-437d-ad2a-c2cb1233ed7f None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:06.905662 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5a768-8fa8-437d-ad2a-c2cb1233ed7f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:06.906854 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f5a768-8fa8-437d-ad2a-c2cb1233ed7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:06.908373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f5a768-8fa8-437d-ad2a-c2cb1233ed7f None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:06.910210 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 523/1037] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:06.915858 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be9242-a44b-47a7-9492-87e798833732 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:07.042800 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be9242-a44b-47a7-9492-87e798833732 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:07.076174 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:07.083419 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe799068-df35-4aab-a502-b5e6d8ba1e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:07.084141 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe799068-df35-4aab-a502-b5e6d8ba1e6b None admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:07.084429 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe799068-df35-4aab-a502-b5e6d8ba1e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:07.084713 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe799068-df35-4aab-a502-b5e6d8ba1e6b None admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:07.085481 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe799068-df35-4aab-a502-b5e6d8ba1e6b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:07.087220 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe799068-df35-4aab-a502-b5e6d8ba1e6b None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:07.111929 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 515/1038] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Jun 27 21:04:06 2022] POST /identity/v3/users => generated 285 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:04:07.412103 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d9ca6-aebe-47c9-a834-4c464af81ee3 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:07.412390 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d9ca6-aebe-47c9-a834-4c464af81ee3 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:07.412669 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d9ca6-aebe-47c9-a834-4c464af81ee3 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:07.414106 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 524/1039] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:04:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:07.419714 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec9d3d-5607-4141-96ab-dcb38af19531 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:07.420003 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec9d3d-5607-4141-96ab-dcb38af19531 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:07.420280 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec9d3d-5607-4141-96ab-dcb38af19531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:07.438276 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-92ec9d3d-5607-4141-96ab-dcb38af19531 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:07.439473 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-92ec9d3d-5607-4141-96ab-dcb38af19531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:07.511003 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 516/1040] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:07 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:07.535178 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d9ca6-aebe-47c9-a834-4c464af81ee3 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:07.559375 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afbb1f6-c883-4403-baee-65dd6fbe6d74 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:07.559680 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afbb1f6-c883-4403-baee-65dd6fbe6d74 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:07.560056 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afbb1f6-c883-4403-baee-65dd6fbe6d74 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:07.561482 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 517/1041] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:04:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:07.567149 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51fcd5-8c26-4e58-8436-b375adeb67e2 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:07.567423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51fcd5-8c26-4e58-8436-b375adeb67e2 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:07.567703 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51fcd5-8c26-4e58-8436-b375adeb67e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:07.587434 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-ff51fcd5-8c26-4e58-8436-b375adeb67e2 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:07.589442 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-ff51fcd5-8c26-4e58-8436-b375adeb67e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:07.658403 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d9ca6-aebe-47c9-a834-4c464af81ee3 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:07.677055 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 518/1042] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:07 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:04:07.698788 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff51fcd5-8c26-4e58-8436-b375adeb67e2 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:07.706941 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e2bfe-8c9b-43a7-8ca7-05636076fda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:07.707829 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e2bfe-8c9b-43a7-8ca7-05636076fda8 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:07.708219 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e2bfe-8c9b-43a7-8ca7-05636076fda8 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:07.708487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e2bfe-8c9b-43a7-8ca7-05636076fda8 None admin] PATH_INFO: `/v3/services/image` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:07.709297 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370e2bfe-8c9b-43a7-8ca7-05636076fda8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:07.710928 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370e2bfe-8c9b-43a7-8ca7-05636076fda8 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:07.724776 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-370e2bfe-8c9b-43a7-8ca7-05636076fda8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 21:04:07.724776 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:07.724776 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:04:07.724776 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:04:07.724776 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:04:07.724776 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:04:07.724776 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:04:07.724776 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:07.724776 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 21:04:07.724776 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:04:07.724776 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:04:07.724776 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:04:07.724776 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:07.724776 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:07.724776 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:07.724776 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:07.724776 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:07.726344 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:07.726344 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:07.726344 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:07.726344 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:07.726344 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:07.726344 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:07.726344 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:04:07.726344 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:04:07.726344 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:04:07.726344 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:04:07.726344 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:07.726344 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:07.726344 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:07.726344 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:07.726344 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:07.726344 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:07.727225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:07.727225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:07.727225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:07.727225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:07.727225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:07.727225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:07.727225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:07.727225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:07.727225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:07.727225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:07.727225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:04:07.727225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:07.727225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 21:04:07.727225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:04:07.727225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 525/1043] 10.222.0.13 () {56 vars in 1105 bytes} [Mon Jun 27 21:04:07 2022] GET /identity/v3/services/image => generated 86 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:07.735022 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e2bfe-8c9b-43a7-8ca7-05636076fda8 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:07.762346 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:07.769694 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:07.827376 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff51fcd5-8c26-4e58-8436-b375adeb67e2 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:07.858434 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e2bfe-8c9b-43a7-8ca7-05636076fda8 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:07.869260 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15021843-93c5-4452-b33e-b78e9ce0631b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:07.870493 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15021843-93c5-4452-b33e-b78e9ce0631b None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:07.871192 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15021843-93c5-4452-b33e-b78e9ce0631b None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:07.871192 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15021843-93c5-4452-b33e-b78e9ce0631b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:07.888644 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15021843-93c5-4452-b33e-b78e9ce0631b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:04:07.896885 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-15021843-93c5-4452-b33e-b78e9ce0631b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 21:04:07.896885 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:07.896885 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:07.896885 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:07.896885 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:07.896885 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:07.896885 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:07.896885 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:07.896885 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:07.896885 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:07.896885 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:07.896885 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:07.896885 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:04:07.896885 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:04:07.896885 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:04:07.896885 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:04:07.897862 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:04:07.897862 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:04:07.897862 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:04:07.897862 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:04:07.897862 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:07.897862 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:07.897862 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:07.897862 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:07.897862 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:07.897862 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:07.897862 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:07.897862 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:07.897862 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:07.897862 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:07.897862 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:07.897862 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:07.898706 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:07.898706 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:07.898706 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:07.898706 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:07.898706 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:04:07.898706 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:04:07.898706 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:04:07.898706 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:04:07.898706 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:04:07.898706 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:04:07.898706 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 21:04:07.898706 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:04:07.898706 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 519/1044] 10.222.0.13 () {56 vars in 1120 bytes} [Mon Jun 27 21:04:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:04:07.904356 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15021843-93c5-4452-b33e-b78e9ce0631b None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:07.909173 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8eeed5-b81a-4ab1-a769-9866354784ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:07.910021 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8eeed5-b81a-4ab1-a769-9866354784ad None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:07.910279 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8eeed5-b81a-4ab1-a769-9866354784ad None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:07.910627 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8eeed5-b81a-4ab1-a769-9866354784ad None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:07.911434 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8eeed5-b81a-4ab1-a769-9866354784ad None admin] RBAC: Adding query filter params (name=image) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:07.911785 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8eeed5-b81a-4ab1-a769-9866354784ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:07.913298 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8eeed5-b81a-4ab1-a769-9866354784ad None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:07.920983 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 526/1045] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:07 2022] GET /identity/v3/services?name=image => generated 122 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:07.927000 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8eeed5-b81a-4ab1-a769-9866354784ad None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:08.029706 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:08.037990 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15021843-93c5-4452-b33e-b78e9ce0631b None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:08.041787 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8eeed5-b81a-4ab1-a769-9866354784ad None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:08.051371 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:08.082283 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112141ba-8815-43f0-ba08-d28ce3e45e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:08.082953 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112141ba-8815-43f0-ba08-d28ce3e45e10 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:08.083221 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112141ba-8815-43f0-ba08-d28ce3e45e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:08.083493 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112141ba-8815-43f0-ba08-d28ce3e45e10 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:08.084201 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112141ba-8815-43f0-ba08-d28ce3e45e10 None admin] RBAC: Adding query filter params (type=image) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:08.084512 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112141ba-8815-43f0-ba08-d28ce3e45e10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:08.084834 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5cea1-bae3-494e-a1d1-c60937af3b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:08.085538 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5cea1-bae3-494e-a1d1-c60937af3b79 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:08.085818 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5cea1-bae3-494e-a1d1-c60937af3b79 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:08.085927 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112141ba-8815-43f0-ba08-d28ce3e45e10 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:08.086095 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5cea1-bae3-494e-a1d1-c60937af3b79 None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:08.086829 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f5cea1-bae3-494e-a1d1-c60937af3b79 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:08.087181 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f5cea1-bae3-494e-a1d1-c60937af3b79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:08.089258 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f5cea1-bae3-494e-a1d1-c60937af3b79 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:08.092015 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 527/1046] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:07 2022] GET /identity/v3/services?type=image => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:08.097586 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112141ba-8815-43f0-ba08-d28ce3e45e10 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:08.099944 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 520/1047] 10.222.0.13 () {56 vars in 1115 bytes} [Mon Jun 27 21:04:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:08.106112 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5cea1-bae3-494e-a1d1-c60937af3b79 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:08.214751 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112141ba-8815-43f0-ba08-d28ce3e45e10 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:08.221648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5cea1-bae3-494e-a1d1-c60937af3b79 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:08.259811 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f837d09e-da4e-4a39-ae17-627120035134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:08.260478 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837d09e-da4e-4a39-ae17-627120035134 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:08.260750 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837d09e-da4e-4a39-ae17-627120035134 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:08.261134 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837d09e-da4e-4a39-ae17-627120035134 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:08.262079 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f837d09e-da4e-4a39-ae17-627120035134 None admin] RBAC: Adding query filter params (interface=public, service_id=54318b6dd7814a98bc9c020496c5b093, region_id=RegionOne) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:08.263685 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f837d09e-da4e-4a39-ae17-627120035134 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:08.265155 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f837d09e-da4e-4a39-ae17-627120035134 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:08.267491 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3687a9c2-1284-4f97-8eae-cfdfdbd67310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:08.268254 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687a9c2-1284-4f97-8eae-cfdfdbd67310 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:08.268535 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687a9c2-1284-4f97-8eae-cfdfdbd67310 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:08.268806 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687a9c2-1284-4f97-8eae-cfdfdbd67310 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:08.279319 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 528/1048] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Jun 27 21:04:08 2022] GET /identity/v3/endpoints?service_id=54318b6dd7814a98bc9c020496c5b093&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:04:08.280847 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f837d09e-da4e-4a39-ae17-627120035134 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:08.283397 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3687a9c2-1284-4f97-8eae-cfdfdbd67310 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:08.284989 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3687a9c2-1284-4f97-8eae-cfdfdbd67310 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:08.303573 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:08.305044 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-3687a9c2-1284-4f97-8eae-cfdfdbd67310 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:08.305044 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:08.305044 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:08.305044 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:08.305044 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:08.305044 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:08.305044 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:08.305044 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:08.305044 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:08.305044 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:08.305044 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:08.305044 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:08.305044 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:04:08.305044 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:04:08.305044 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:04:08.305948 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:04:08.305948 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:08.305948 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:08.305948 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:08.305948 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:08.305948 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:08.305948 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:08.305948 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:08.305948 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:08.305948 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:08.305948 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:08.305948 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:08.305948 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:08.305948 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:08.305948 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:08.305948 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:08.305948 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:08.312459 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:04:08.312459 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:04:08.312459 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:08.312459 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:04:08.312459 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 521/1049] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:04:08.312768 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3687a9c2-1284-4f97-8eae-cfdfdbd67310 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:08.346848 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:08.413515 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f837d09e-da4e-4a39-ae17-627120035134 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:08.461597 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3687a9c2-1284-4f97-8eae-cfdfdbd67310 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:08.463406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f0901-9468-46aa-b206-89e002911a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:08.465101 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f0901-9468-46aa-b206-89e002911a13 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:08.465528 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f0901-9468-46aa-b206-89e002911a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:08.465943 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f0901-9468-46aa-b206-89e002911a13 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:08.466776 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938f0901-9468-46aa-b206-89e002911a13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:08.468311 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938f0901-9468-46aa-b206-89e002911a13 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:08.477949 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 529/1050] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Jun 27 21:04:08 2022] GET /identity/v3/services => generated 1957 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:04:08.518194 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe869ee8-85aa-4523-844d-f93f148286f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:08.518796 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe869ee8-85aa-4523-844d-f93f148286f9 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:08.519105 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe869ee8-85aa-4523-844d-f93f148286f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:08.519342 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe869ee8-85aa-4523-844d-f93f148286f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:08.520089 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe869ee8-85aa-4523-844d-f93f148286f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:08.520396 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe869ee8-85aa-4523-844d-f93f148286f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:08.521836 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe869ee8-85aa-4523-844d-f93f148286f9 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:08.536613 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 522/1051] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:04:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:04:08.542657 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f0901-9468-46aa-b206-89e002911a13 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:08.684752 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:08.711652 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f0901-9468-46aa-b206-89e002911a13 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:08.753156 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f4bea-7989-4e17-aabd-8a994c9f4aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:08.753963 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f4bea-7989-4e17-aabd-8a994c9f4aee None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:08.754248 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f4bea-7989-4e17-aabd-8a994c9f4aee None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:08.754525 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f4bea-7989-4e17-aabd-8a994c9f4aee None admin] PATH_INFO: `/v3/users/nova` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:08.763750 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355f4bea-7989-4e17-aabd-8a994c9f4aee None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:08.765276 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355f4bea-7989-4e17-aabd-8a994c9f4aee None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:08.781322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-355f4bea-7989-4e17-aabd-8a994c9f4aee None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 21:04:08.781322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:08.781322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:08.781322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:08.781322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:08.781322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:08.781322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:08.781322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:08.781322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:08.781322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:08.781322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:08.781322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:08.781322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:04:08.781322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:04:08.781322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:04:08.781322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:04:08.782361 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:08.782361 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:08.782361 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:04:08.782361 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:04:08.782361 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:04:08.782361 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:04:08.782361 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:08.782361 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:08.782361 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:08.782361 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:08.782361 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:08.782361 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:08.782361 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:08.782361 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:08.782361 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:08.782361 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:08.782361 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:08.782361 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:08.788402 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:08.788402 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:08.788402 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:04:08.788402 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:04:08.788402 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:04:08.788402 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:04:08.788402 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:04:08.788402 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:04:08.788402 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 21:04:08.788402 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:04:08.788402 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 530/1052] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Jun 27 21:04:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:08.790699 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe869ee8-85aa-4523-844d-f93f148286f9 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:08.896799 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:08.926046 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe869ee8-85aa-4523-844d-f93f148286f9 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:08.980894 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60239428-9adb-43b5-a33f-ed30b86ad952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:08.981624 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60239428-9adb-43b5-a33f-ed30b86ad952 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:08.982039 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60239428-9adb-43b5-a33f-ed30b86ad952 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:08.982365 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60239428-9adb-43b5-a33f-ed30b86ad952 None admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:08.983867 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60239428-9adb-43b5-a33f-ed30b86ad952 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:08.983867 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60239428-9adb-43b5-a33f-ed30b86ad952 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:08.984977 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60239428-9adb-43b5-a33f-ed30b86ad952 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:09.009132 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 523/1053] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Jun 27 21:04:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:09.017786 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f4bea-7989-4e17-aabd-8a994c9f4aee None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:09.139303 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f4bea-7989-4e17-aabd-8a994c9f4aee None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:09.176820 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40497645-873b-44af-9247-588b0205c4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:09.177686 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40497645-873b-44af-9247-588b0205c4be None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:09.177955 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40497645-873b-44af-9247-588b0205c4be None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:09.178237 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40497645-873b-44af-9247-588b0205c4be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:09.191780 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40497645-873b-44af-9247-588b0205c4be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:09.193404 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40497645-873b-44af-9247-588b0205c4be None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:09.196770 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:09.219282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-40497645-873b-44af-9247-588b0205c4be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:09.219282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:09.219282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:09.219282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:09.219282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:09.219282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:09.219282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:09.219282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:09.219282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:09.219282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:09.219282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:09.219282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:09.219282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:04:09.219282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:04:09.219282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:04:09.220279 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:04:09.220279 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:09.220279 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:09.220279 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:09.220279 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:09.220279 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:09.220279 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:09.220279 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:09.220279 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:09.220279 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:09.220279 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:09.220279 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:09.220279 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:09.220279 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:09.220279 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:09.220279 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:09.220279 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:09.221728 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:04:09.221728 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:04:09.221728 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:09.221728 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:04:09.225218 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 531/1054] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:09.227675 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60239428-9adb-43b5-a33f-ed30b86ad952 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:09.313750 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03171cc0-e540-43bd-a1fc-c0f8c9a0d2ad None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:09.314042 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03171cc0-e540-43bd-a1fc-c0f8c9a0d2ad None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:09.314301 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03171cc0-e540-43bd-a1fc-c0f8c9a0d2ad None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:09.315717 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 532/1055] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:04:09.321375 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c29af8-b621-4d7c-9d7f-5c9f60774cd3 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:09.321646 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c29af8-b621-4d7c-9d7f-5c9f60774cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:09.321961 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c29af8-b621-4d7c-9d7f-5c9f60774cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:09.343037 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-41c29af8-b621-4d7c-9d7f-5c9f60774cd3 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:09.344693 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-41c29af8-b621-4d7c-9d7f-5c9f60774cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:09.375968 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60239428-9adb-43b5-a33f-ed30b86ad952 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:09.420020 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b60cce-2dd7-4b91-bfbc-91ee7120f804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:09.420340 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 533/1056] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:04:09.421440 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b60cce-2dd7-4b91-bfbc-91ee7120f804 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:09.421891 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b60cce-2dd7-4b91-bfbc-91ee7120f804 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:09.422303 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b60cce-2dd7-4b91-bfbc-91ee7120f804 None admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:09.423834 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b60cce-2dd7-4b91-bfbc-91ee7120f804 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:09.424291 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b60cce-2dd7-4b91-bfbc-91ee7120f804 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:09.426386 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b60cce-2dd7-4b91-bfbc-91ee7120f804 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:09.437659 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:09.446444 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 524/1057] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:04:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:09.448597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c29af8-b621-4d7c-9d7f-5c9f60774cd3 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:09.450018 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b60cce-2dd7-4b91-bfbc-91ee7120f804 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:09.572877 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c29af8-b621-4d7c-9d7f-5c9f60774cd3 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:09.579304 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b60cce-2dd7-4b91-bfbc-91ee7120f804 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:09.608843 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:09.623883 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e619d6f-94e3-4424-a321-acc2d6beec3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:09.624696 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e619d6f-94e3-4424-a321-acc2d6beec3f None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:09.625008 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e619d6f-94e3-4424-a321-acc2d6beec3f None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:09.625454 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e619d6f-94e3-4424-a321-acc2d6beec3f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:09.630576 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1212e377-8f88-4d7f-b272-0366b1bf3562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:09.631393 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1212e377-8f88-4d7f-b272-0366b1bf3562 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:09.631679 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1212e377-8f88-4d7f-b272-0366b1bf3562 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:09.632211 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1212e377-8f88-4d7f-b272-0366b1bf3562 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:09.633142 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e619d6f-94e3-4424-a321-acc2d6beec3f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:09.634496 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e619d6f-94e3-4424-a321-acc2d6beec3f None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:09.644906 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1212e377-8f88-4d7f-b272-0366b1bf3562 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:04:09.649399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-0e619d6f-94e3-4424-a321-acc2d6beec3f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:04:09.649399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:09.649399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:09.649399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:09.649399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:09.649399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:09.649399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:09.649399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:09.649399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:09.649399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:09.649399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:09.649399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:09.649399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:04:09.649399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:04:09.649399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:04:09.650450 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:04:09.650450 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:09.650450 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:09.650450 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:09.650450 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:09.650450 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:09.650450 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:09.650450 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:09.650450 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:09.650450 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:09.650450 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:09.650450 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:09.650450 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:09.650450 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:09.650450 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:09.650450 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:09.650450 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:09.651622 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:04:09.651622 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:04:09.651622 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:04:09.651622 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:04:09.651622 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:04:09.651622 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:04:09.651622 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:04:09.651622 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:04:09.651622 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 534/1058] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:04:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:09.655128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-1212e377-8f88-4d7f-b272-0366b1bf3562 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:04:09.655128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:09.655128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:09.655128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:09.655128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:09.655128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:09.655128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:09.655128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:09.655128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:09.655128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:09.655128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:09.655128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:09.655128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:04:09.655128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:04:09.655128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:04:09.655128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:04:09.656104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:04:09.656104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:04:09.656104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:04:09.656104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:04:09.656104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:09.656104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:09.656104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:09.656104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:09.656104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:09.656104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:09.656104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:09.656104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:09.656104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:09.656104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:09.656104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:09.656104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:09.656987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:09.656987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:09.656987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:09.656987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:09.656987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:04:09.656987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:04:09.656987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:04:09.656987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:04:09.656987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:04:09.656987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:04:09.656987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:04:09.656987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:04:09.656987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 525/1059] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 27 21:04:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:04:09.658723 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e619d6f-94e3-4424-a321-acc2d6beec3f None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:09.662292 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1212e377-8f88-4d7f-b272-0366b1bf3562 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:09.742538 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:09.773438 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e619d6f-94e3-4424-a321-acc2d6beec3f None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:09.791556 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1212e377-8f88-4d7f-b272-0366b1bf3562 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:09.810596 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb058f6-b5bf-49a2-8515-248cd62a1a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:09.811257 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb058f6-b5bf-49a2-8515-248cd62a1a1c None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:09.811550 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb058f6-b5bf-49a2-8515-248cd62a1a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:09.811820 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb058f6-b5bf-49a2-8515-248cd62a1a1c None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:09.812616 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb058f6-b5bf-49a2-8515-248cd62a1a1c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:09.812945 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb058f6-b5bf-49a2-8515-248cd62a1a1c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:09.814404 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb058f6-b5bf-49a2-8515-248cd62a1a1c None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:09.827430 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 535/1060] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 27 21:04:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:09.828967 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede79ac8-2a99-482c-9696-7107fd7457b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:09.829647 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede79ac8-2a99-482c-9696-7107fd7457b9 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:09.829931 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede79ac8-2a99-482c-9696-7107fd7457b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:09.830203 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede79ac8-2a99-482c-9696-7107fd7457b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:09.831009 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede79ac8-2a99-482c-9696-7107fd7457b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:09.831314 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede79ac8-2a99-482c-9696-7107fd7457b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:09.832764 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede79ac8-2a99-482c-9696-7107fd7457b9 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:09.833375 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb058f6-b5bf-49a2-8515-248cd62a1a1c None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:09.846623 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 526/1061] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:04:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:04:09.847645 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede79ac8-2a99-482c-9696-7107fd7457b9 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:09.887410 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:09.946031 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb058f6-b5bf-49a2-8515-248cd62a1a1c None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:09.953736 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede79ac8-2a99-482c-9696-7107fd7457b9 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:09.976967 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:09.985672 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220cef27-f709-46de-a3eb-51b75f20347e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:09.986571 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220cef27-f709-46de-a3eb-51b75f20347e None admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:09.986853 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220cef27-f709-46de-a3eb-51b75f20347e None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:09.987126 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220cef27-f709-46de-a3eb-51b75f20347e None admin] PATH_INFO: `/v3/projects/334438c929c44e489e9a3b93e6d37231/users/1366958034d540b4ba2feb421a6afabc/roles/b3dcc5de8383479d995a9bc38d8a8ef2` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:09.989323 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220cef27-f709-46de-a3eb-51b75f20347e None admin] RBAC: Authorizing `identity:create_grant(project_id=334438c929c44e489e9a3b93e6d37231, user_id=1366958034d540b4ba2feb421a6afabc, role_id=b3dcc5de8383479d995a9bc38d8a8ef2)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:09.989432 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d4397-249b-4a52-8f48-e7bdfe71afb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:09.990245 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d4397-249b-4a52-8f48-e7bdfe71afb5 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:09.990528 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d4397-249b-4a52-8f48-e7bdfe71afb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:09.990772 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220cef27-f709-46de-a3eb-51b75f20347e None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:09.990869 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d4397-249b-4a52-8f48-e7bdfe71afb5 None admin] PATH_INFO: `/v3/users/87f38a9c73404af7b8cff77ab652599a` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:10.004987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d4397-249b-4a52-8f48-e7bdfe71afb5 None admin] RBAC: Authorizing `identity:get_user(user_id=87f38a9c73404af7b8cff77ab652599a)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:10.006373 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d4397-249b-4a52-8f48-e7bdfe71afb5 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:10.015978 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 527/1063] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:04:09 2022] GET /identity/v3/users/87f38a9c73404af7b8cff77ab652599a => generated 285 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:04:10.016212 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 536/1063] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 27 21:04:09 2022] PUT /identity/v3/projects/334438c929c44e489e9a3b93e6d37231/users/1366958034d540b4ba2feb421a6afabc/roles/b3dcc5de8383479d995a9bc38d8a8ef2 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:04:10.016299 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d4397-249b-4a52-8f48-e7bdfe71afb5 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:10.133261 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d4397-249b-4a52-8f48-e7bdfe71afb5 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:10.173682 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db00110-4080-4012-afe2-a4c05b5a7632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:10.174345 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db00110-4080-4012-afe2-a4c05b5a7632 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:10.174629 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db00110-4080-4012-afe2-a4c05b5a7632 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:10.174907 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db00110-4080-4012-afe2-a4c05b5a7632 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:10.226526 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:10.248575 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db00110-4080-4012-afe2-a4c05b5a7632 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:10.250142 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db00110-4080-4012-afe2-a4c05b5a7632 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:10.355647 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 528/1064] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:04:10 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 21:04:10.361256 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220cef27-f709-46de-a3eb-51b75f20347e None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:10.466424 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220cef27-f709-46de-a3eb-51b75f20347e None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:10.471553 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:10.509526 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f0b5a8-4234-4e35-937f-ef2cf788c422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:10.510427 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0b5a8-4234-4e35-937f-ef2cf788c422 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:10.510704 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0b5a8-4234-4e35-937f-ef2cf788c422 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:10.510978 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0b5a8-4234-4e35-937f-ef2cf788c422 None admin] PATH_INFO: `/v3/projects/0fcb2a9eb1314203b57d999c396a0f9d` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:10.512199 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f0b5a8-4234-4e35-937f-ef2cf788c422 None admin] RBAC: Authorizing `identity:get_project(project_id=0fcb2a9eb1314203b57d999c396a0f9d)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:10.513686 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f0b5a8-4234-4e35-937f-ef2cf788c422 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:10.515602 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 537/1065] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:04:10 2022] GET /identity/v3/projects/0fcb2a9eb1314203b57d999c396a0f9d => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:10.521649 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db00110-4080-4012-afe2-a4c05b5a7632 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:10.608047 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:10.629587 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db00110-4080-4012-afe2-a4c05b5a7632 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:10.639682 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0750b4-6395-4de5-985c-7f5eef57cbe8 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:10.639981 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0750b4-6395-4de5-985c-7f5eef57cbe8 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:10.640229 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0750b4-6395-4de5-985c-7f5eef57cbe8 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:10.643054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 538/1066] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:04:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:04:10.647042 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e06ae4-3330-4bee-8825-cf0b64cc9d96 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:10.647343 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e06ae4-3330-4bee-8825-cf0b64cc9d96 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:10.647598 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e06ae4-3330-4bee-8825-cf0b64cc9d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:10.664331 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-31e06ae4-3330-4bee-8825-cf0b64cc9d96 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:10.665588 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-31e06ae4-3330-4bee-8825-cf0b64cc9d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:10.671872 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4720a7-f8c0-4c20-a7a2-b33331f9e7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:10.672543 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4720a7-f8c0-4c20-a7a2-b33331f9e7b9 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:10.672826 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4720a7-f8c0-4c20-a7a2-b33331f9e7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:10.673131 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4720a7-f8c0-4c20-a7a2-b33331f9e7b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:10.673965 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4720a7-f8c0-4c20-a7a2-b33331f9e7b9 None admin] RBAC: Adding query filter params (role.id=d336dd7b746c4488b33289c00302e4d5, scope.project.id=0fcb2a9eb1314203b57d999c396a0f9d, user.id=87f38a9c73404af7b8cff77ab652599a) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:10.674288 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4720a7-f8c0-4c20-a7a2-b33331f9e7b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:10.675719 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4720a7-f8c0-4c20-a7a2-b33331f9e7b9 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:10.688979 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 529/1067] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:04:10 2022] GET /identity/v3/role_assignments?user.id=87f38a9c73404af7b8cff77ab652599a&scope.project.id=0fcb2a9eb1314203b57d999c396a0f9d&role.id=d336dd7b746c4488b33289c00302e4d5 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:10.751000 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 539/1068] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:10 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:04:10.774200 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4720a7-f8c0-4c20-a7a2-b33331f9e7b9 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:10.882506 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4720a7-f8c0-4c20-a7a2-b33331f9e7b9 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:10.919448 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fbcf6a-3c64-43dd-8e7f-6ff32821c301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:10.920247 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fbcf6a-3c64-43dd-8e7f-6ff32821c301 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:10.920527 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fbcf6a-3c64-43dd-8e7f-6ff32821c301 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:10.920785 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fbcf6a-3c64-43dd-8e7f-6ff32821c301 None admin] PATH_INFO: `/v3/services/image` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:10.921488 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fbcf6a-3c64-43dd-8e7f-6ff32821c301 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:10.922860 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fbcf6a-3c64-43dd-8e7f-6ff32821c301 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:10.934599 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-70fbcf6a-3c64-43dd-8e7f-6ff32821c301 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 21:04:10.934599 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:10.934599 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:04:10.934599 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:04:10.934599 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:04:10.934599 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:04:10.934599 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:04:10.934599 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:10.934599 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 21:04:10.934599 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:04:10.934599 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:04:10.934599 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:04:10.934599 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:10.934599 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:10.934599 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:10.934599 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:10.934599 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:10.935613 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:10.935613 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:10.935613 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:10.935613 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:10.935613 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:10.935613 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:10.935613 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:04:10.935613 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:04:10.935613 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:04:10.935613 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:04:10.935613 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:10.935613 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:10.935613 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:10.935613 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:10.935613 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:10.935613 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:10.936481 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:10.936481 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:10.936481 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:10.936481 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:10.936481 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:10.936481 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:10.936481 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:10.936481 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:10.936481 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:10.936481 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:10.936481 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:04:10.936481 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:04:10.936481 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 27 21:04:10.936481 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:04:10.936481 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 530/1069] 10.222.0.13 () {56 vars in 1105 bytes} [Mon Jun 27 21:04:10 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:04:10.941834 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e06ae4-3330-4bee-8825-cf0b64cc9d96 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:11.052940 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e06ae4-3330-4bee-8825-cf0b64cc9d96 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:11.054973 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:11.091742 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3948cb-b2bb-4d74-8953-7a58a7b419bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:11.092417 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3948cb-b2bb-4d74-8953-7a58a7b419bd None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:11.092697 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3948cb-b2bb-4d74-8953-7a58a7b419bd None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:11.092997 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3948cb-b2bb-4d74-8953-7a58a7b419bd None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:11.093718 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3948cb-b2bb-4d74-8953-7a58a7b419bd None admin] RBAC: Adding query filter params (name=image) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:11.094026 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3948cb-b2bb-4d74-8953-7a58a7b419bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:11.095411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3948cb-b2bb-4d74-8953-7a58a7b419bd None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:11.101319 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 540/1070] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:11.107417 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fbcf6a-3c64-43dd-8e7f-6ff32821c301 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:11.120403 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:11.220100 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fbcf6a-3c64-43dd-8e7f-6ff32821c301 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:11.255507 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b776980-05b8-47b2-b725-a4e7f5f6def0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:11.256177 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b776980-05b8-47b2-b725-a4e7f5f6def0 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:11.256457 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b776980-05b8-47b2-b725-a4e7f5f6def0 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:11.256731 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b776980-05b8-47b2-b725-a4e7f5f6def0 None admin] PATH_INFO: `/v3/services` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:11.257466 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b776980-05b8-47b2-b725-a4e7f5f6def0 None admin] RBAC: Adding query filter params (type=image) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:11.257771 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b776980-05b8-47b2-b725-a4e7f5f6def0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:11.259173 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b776980-05b8-47b2-b725-a4e7f5f6def0 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:11.264457 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 531/1071] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:11 2022] GET /identity/v3/services?type=image => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:11.269975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3948cb-b2bb-4d74-8953-7a58a7b419bd None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:11.372101 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3948cb-b2bb-4d74-8953-7a58a7b419bd None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:11.406547 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abfad1a-946d-4cc4-893b-032d5bb518e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:11.407207 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abfad1a-946d-4cc4-893b-032d5bb518e3 None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:11.407484 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abfad1a-946d-4cc4-893b-032d5bb518e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:11.407756 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abfad1a-946d-4cc4-893b-032d5bb518e3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:11.408502 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abfad1a-946d-4cc4-893b-032d5bb518e3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:11.409901 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abfad1a-946d-4cc4-893b-032d5bb518e3 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:11.430718 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 541/1072] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Jun 27 21:04:11 2022] POST /identity/v3/endpoints => generated 327 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:04:12.055285 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d95c28-180f-4ff6-9359-3c37c8a680f9 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:12.055800 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d95c28-180f-4ff6-9359-3c37c8a680f9 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:12.055911 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d95c28-180f-4ff6-9359-3c37c8a680f9 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:12.057479 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 532/1073] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:04:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:12.062679 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba2599-8674-4750-8d25-f9ff70d4179d None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:12.063099 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba2599-8674-4750-8d25-f9ff70d4179d None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:12.063273 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba2599-8674-4750-8d25-f9ff70d4179d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:12.081846 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-14ba2599-8674-4750-8d25-f9ff70d4179d None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:12.083049 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-14ba2599-8674-4750-8d25-f9ff70d4179d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:12.102571 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:12.164108 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 542/1074] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:12.188201 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d95c28-180f-4ff6-9359-3c37c8a680f9 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:12.265731 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:12.298915 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d95c28-180f-4ff6-9359-3c37c8a680f9 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:12.334592 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d970010-68b7-49d6-b3c8-479b6bc63088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:12.335405 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d970010-68b7-49d6-b3c8-479b6bc63088 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:12.335697 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d970010-68b7-49d6-b3c8-479b6bc63088 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:12.335973 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d970010-68b7-49d6-b3c8-479b6bc63088 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:12.347281 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d970010-68b7-49d6-b3c8-479b6bc63088 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:04:12.355916 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-2d970010-68b7-49d6-b3c8-479b6bc63088 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 21:04:12.355916 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:12.355916 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:12.355916 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:12.355916 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:12.355916 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:12.355916 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:12.355916 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:12.355916 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:12.355916 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:12.355916 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:12.355916 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:12.355916 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:04:12.355916 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:04:12.355916 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:04:12.355916 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:04:12.356874 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:04:12.356874 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:04:12.356874 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:04:12.356874 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:04:12.356874 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:12.356874 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:12.356874 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:12.356874 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:12.356874 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:12.356874 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:12.356874 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:12.356874 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:12.356874 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:12.356874 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:12.356874 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:12.356874 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:12.358110 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:12.358110 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:12.358110 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:12.358110 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:12.358110 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:04:12.358110 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:04:12.358110 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:04:12.358110 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:04:12.358110 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:04:12.358110 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:04:12.358110 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 27 21:04:12.358110 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:04:12.361320 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 533/1075] 10.222.0.13 () {56 vars in 1120 bytes} [Mon Jun 27 21:04:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:12.363295 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba2599-8674-4750-8d25-f9ff70d4179d None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:12.469706 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba2599-8674-4750-8d25-f9ff70d4179d None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:12.505888 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2966eba9-ed68-455d-97c4-edad6723d155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:12.506556 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966eba9-ed68-455d-97c4-edad6723d155 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:12.506840 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966eba9-ed68-455d-97c4-edad6723d155 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:12.507110 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966eba9-ed68-455d-97c4-edad6723d155 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:12.507919 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2966eba9-ed68-455d-97c4-edad6723d155 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:12.508226 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2966eba9-ed68-455d-97c4-edad6723d155 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:12.509760 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2966eba9-ed68-455d-97c4-edad6723d155 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:12.524007 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 543/1076] 10.222.0.13 () {56 vars in 1115 bytes} [Mon Jun 27 21:04:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:12.526016 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d970010-68b7-49d6-b3c8-479b6bc63088 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:12.646690 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d970010-68b7-49d6-b3c8-479b6bc63088 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:12.679358 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:12.685953 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e75a780-dec9-49a8-b235-5127ddf5b77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:12.686767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75a780-dec9-49a8-b235-5127ddf5b77b None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:12.687042 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75a780-dec9-49a8-b235-5127ddf5b77b None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:12.687308 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75a780-dec9-49a8-b235-5127ddf5b77b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:12.687696 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71e0ea-913d-4dce-9561-9c79f9640c3e None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:12.688088 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71e0ea-913d-4dce-9561-9c79f9640c3e None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:12.688370 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71e0ea-913d-4dce-9561-9c79f9640c3e None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:12.690115 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 544/1077] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:04:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:12.695160 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48555d8-7e08-45b0-8628-6570119bcef5 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:12.695645 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48555d8-7e08-45b0-8628-6570119bcef5 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:12.696114 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48555d8-7e08-45b0-8628-6570119bcef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:12.699330 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e75a780-dec9-49a8-b235-5127ddf5b77b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:12.700795 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e75a780-dec9-49a8-b235-5127ddf5b77b None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:12.714660 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-e48555d8-7e08-45b0-8628-6570119bcef5 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:12.715842 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-e48555d8-7e08-45b0-8628-6570119bcef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:12.719577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:12.720755 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-7e75a780-dec9-49a8-b235-5127ddf5b77b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:12.720755 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:12.720755 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:12.720755 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:12.720755 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:12.720755 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:12.720755 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:12.720755 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:12.720755 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:12.720755 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:12.720755 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:12.720755 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:12.720755 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:04:12.720755 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:04:12.720755 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:04:12.721805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:04:12.721805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:12.721805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:12.721805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:12.721805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:12.721805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:12.721805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:12.721805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:12.721805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:12.721805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:12.721805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:12.721805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:12.721805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:12.721805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:12.721805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:12.721805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:12.721805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:12.723180 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:04:12.723180 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:04:12.723180 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:12.723180 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:04:12.723180 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 534/1078] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:12.729906 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e75a780-dec9-49a8-b235-5127ddf5b77b None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:12.800987 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 545/1079] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:04:12.818839 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48555d8-7e08-45b0-8628-6570119bcef5 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:12.839616 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e75a780-dec9-49a8-b235-5127ddf5b77b None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:12.876556 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b32c7-7ddb-4c6b-80c2-85305af4e28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:12.877332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b32c7-7ddb-4c6b-80c2-85305af4e28a None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:12.877538 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b32c7-7ddb-4c6b-80c2-85305af4e28a None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:12.877814 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b32c7-7ddb-4c6b-80c2-85305af4e28a None admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:12.878545 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b32c7-7ddb-4c6b-80c2-85305af4e28a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:12.878854 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b32c7-7ddb-4c6b-80c2-85305af4e28a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:12.880235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b32c7-7ddb-4c6b-80c2-85305af4e28a None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:12.893182 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 535/1080] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:04:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:12.898883 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b32c7-7ddb-4c6b-80c2-85305af4e28a None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:12.927851 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48555d8-7e08-45b0-8628-6570119bcef5 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:12.949783 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:12.965906 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38297d7-ab69-47ed-8beb-6420c3d43511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:12.966820 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38297d7-ab69-47ed-8beb-6420c3d43511 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:12.967217 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38297d7-ab69-47ed-8beb-6420c3d43511 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:12.967494 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38297d7-ab69-47ed-8beb-6420c3d43511 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:12.976563 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:12.980597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38297d7-ab69-47ed-8beb-6420c3d43511 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:04:12.989850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-c38297d7-ab69-47ed-8beb-6420c3d43511 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:04:12.989850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:12.989850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:12.989850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:12.989850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:12.989850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:12.989850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:12.989850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:12.989850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:12.989850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:12.989850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:12.989850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:12.989850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:04:12.989850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:04:12.989850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:04:12.989850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:04:12.990830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:04:12.990830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:04:12.990830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:04:12.990830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:04:12.990830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:12.990830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:12.990830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:12.990830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:12.990830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:12.990830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:12.990830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:12.990830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:12.990830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:12.990830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:12.990830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:12.990830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:12.991716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:12.991716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:12.991716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:12.991716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:12.991716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:04:12.991716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:04:12.991716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:04:12.991716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:04:12.991716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:04:12.991716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:04:12.991716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:04:12.991716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:04:12.991716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 546/1081] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 27 21:04:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:04:12.998002 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38297d7-ab69-47ed-8beb-6420c3d43511 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:13.009480 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b32c7-7ddb-4c6b-80c2-85305af4e28a None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:13.052678 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94159092-c643-4d5c-a614-efd79e58cdb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:13.053589 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94159092-c643-4d5c-a614-efd79e58cdb7 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:13.053874 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94159092-c643-4d5c-a614-efd79e58cdb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:13.054146 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94159092-c643-4d5c-a614-efd79e58cdb7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:13.061886 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94159092-c643-4d5c-a614-efd79e58cdb7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:13.063340 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94159092-c643-4d5c-a614-efd79e58cdb7 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:13.079118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-94159092-c643-4d5c-a614-efd79e58cdb7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 27 21:04:13.079118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:13.079118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:13.079118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:13.079118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:13.079118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:13.079118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:13.079118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:13.079118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:13.079118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:13.079118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:13.079118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:13.079118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:04:13.079118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:04:13.079118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:04:13.079118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:04:13.080128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:13.080128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:13.080128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:04:13.080128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:04:13.080128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:04:13.080128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:04:13.080128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:13.080128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:13.080128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:13.080128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:13.080128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:13.080128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:13.080128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:13.080128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:13.080128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:13.080128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:13.080128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:13.080128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:13.081136 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:13.081136 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:13.081136 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:04:13.081136 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:04:13.081136 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:04:13.081136 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:04:13.081136 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:04:13.081136 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:04:13.081136 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 27 21:04:13.081136 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:04:13.085179 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 536/1082] 10.222.0.13 () {56 vars in 1093 bytes} [Mon Jun 27 21:04:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:13.086074 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94159092-c643-4d5c-a614-efd79e58cdb7 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:13.111356 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38297d7-ab69-47ed-8beb-6420c3d43511 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:13.152741 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858dcd8a-bd79-455f-9dcf-8575e4710796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:13.153671 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858dcd8a-bd79-455f-9dcf-8575e4710796 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:13.154117 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858dcd8a-bd79-455f-9dcf-8575e4710796 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:13.154523 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858dcd8a-bd79-455f-9dcf-8575e4710796 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:13.155425 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858dcd8a-bd79-455f-9dcf-8575e4710796 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:13.155729 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858dcd8a-bd79-455f-9dcf-8575e4710796 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:13.157153 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858dcd8a-bd79-455f-9dcf-8575e4710796 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:13.171251 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 547/1083] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:04:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:04:13.172367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858dcd8a-bd79-455f-9dcf-8575e4710796 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:13.202497 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94159092-c643-4d5c-a614-efd79e58cdb7 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:13.204872 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:13.238932 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d61febb-2d7e-4099-bfe8-ddcd399cbbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:13.240607 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d61febb-2d7e-4099-bfe8-ddcd399cbbe5 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:13.241330 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d61febb-2d7e-4099-bfe8-ddcd399cbbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:13.241763 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d61febb-2d7e-4099-bfe8-ddcd399cbbe5 None admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:13.242661 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:13.244103 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d61febb-2d7e-4099-bfe8-ddcd399cbbe5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:13.244603 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d61febb-2d7e-4099-bfe8-ddcd399cbbe5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:13.246588 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d61febb-2d7e-4099-bfe8-ddcd399cbbe5 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:13.270714 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 537/1084] 10.222.0.13 () {56 vars in 1133 bytes} [Mon Jun 27 21:04:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:04:13.276837 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d61febb-2d7e-4099-bfe8-ddcd399cbbe5 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:13.286009 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858dcd8a-bd79-455f-9dcf-8575e4710796 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:13.321265 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba99e5b6-8a1c-4bb8-88f3-37e36d14fd2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:13.322216 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba99e5b6-8a1c-4bb8-88f3-37e36d14fd2f None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:13.322504 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba99e5b6-8a1c-4bb8-88f3-37e36d14fd2f None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:13.323026 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba99e5b6-8a1c-4bb8-88f3-37e36d14fd2f None admin] PATH_INFO: `/v3/users/87f38a9c73404af7b8cff77ab652599a` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:13.323981 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba99e5b6-8a1c-4bb8-88f3-37e36d14fd2f None admin] RBAC: Authorizing `identity:get_user(user_id=87f38a9c73404af7b8cff77ab652599a)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:13.325508 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba99e5b6-8a1c-4bb8-88f3-37e36d14fd2f None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:13.331877 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 548/1085] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:04:13 2022] GET /identity/v3/users/87f38a9c73404af7b8cff77ab652599a => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:13.332873 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba99e5b6-8a1c-4bb8-88f3-37e36d14fd2f None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:13.392511 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d61febb-2d7e-4099-bfe8-ddcd399cbbe5 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:13.428538 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62117d17-5c3c-4d4c-a7f4-10610971cb78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:13.429235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62117d17-5c3c-4d4c-a7f4-10610971cb78 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:13.429518 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62117d17-5c3c-4d4c-a7f4-10610971cb78 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:13.429794 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62117d17-5c3c-4d4c-a7f4-10610971cb78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:13.446396 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba99e5b6-8a1c-4bb8-88f3-37e36d14fd2f None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:13.462339 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:13.477641 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:13.484888 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d052fc7-17c4-42cf-8fb7-4b77b72e703c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:13.485793 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d052fc7-17c4-42cf-8fb7-4b77b72e703c None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:13.486285 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d052fc7-17c4-42cf-8fb7-4b77b72e703c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:13.486707 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d052fc7-17c4-42cf-8fb7-4b77b72e703c None admin] PATH_INFO: `/v3/projects/0fcb2a9eb1314203b57d999c396a0f9d` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:13.487897 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d052fc7-17c4-42cf-8fb7-4b77b72e703c None admin] RBAC: Authorizing `identity:get_project(project_id=0fcb2a9eb1314203b57d999c396a0f9d)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:13.489393 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d052fc7-17c4-42cf-8fb7-4b77b72e703c None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:13.491883 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 549/1086] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:04:13 2022] GET /identity/v3/projects/0fcb2a9eb1314203b57d999c396a0f9d => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:04:13.492308 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c562ae-3fe9-4999-8fe2-1d3d136940c0 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:13.492592 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c562ae-3fe9-4999-8fe2-1d3d136940c0 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:13.492867 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c562ae-3fe9-4999-8fe2-1d3d136940c0 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:13.494530 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 550/1087] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:04:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:13.496816 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c562ae-3fe9-4999-8fe2-1d3d136940c0 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:13.509005 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62117d17-5c3c-4d4c-a7f4-10610971cb78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:13.510645 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62117d17-5c3c-4d4c-a7f4-10610971cb78 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:13.599292 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c562ae-3fe9-4999-8fe2-1d3d136940c0 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:13.609862 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 538/1088] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:04:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 21:04:13.609862 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9604d72-1bfe-4a47-93d3-6f4224cdd3e5 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:13.609862 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9604d72-1bfe-4a47-93d3-6f4224cdd3e5 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:13.609862 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9604d72-1bfe-4a47-93d3-6f4224cdd3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:13.627520 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-c9604d72-1bfe-4a47-93d3-6f4224cdd3e5 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:13.628730 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-c9604d72-1bfe-4a47-93d3-6f4224cdd3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:13.641103 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ecc2f-6cb9-4b4b-966e-bfa882c9875c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:13.641996 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ecc2f-6cb9-4b4b-966e-bfa882c9875c None admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:13.642274 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ecc2f-6cb9-4b4b-966e-bfa882c9875c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:13.642548 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ecc2f-6cb9-4b4b-966e-bfa882c9875c None admin] PATH_INFO: `/v3/projects/0fcb2a9eb1314203b57d999c396a0f9d/users/87f38a9c73404af7b8cff77ab652599a/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:13.644569 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7ecc2f-6cb9-4b4b-966e-bfa882c9875c None admin] RBAC: Authorizing `identity:create_grant(project_id=0fcb2a9eb1314203b57d999c396a0f9d, user_id=87f38a9c73404af7b8cff77ab652599a, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:13.646048 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7ecc2f-6cb9-4b4b-966e-bfa882c9875c None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:13.659615 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 551/1089] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 27 21:04:13 2022] PUT /identity/v3/projects/0fcb2a9eb1314203b57d999c396a0f9d/users/87f38a9c73404af7b8cff77ab652599a/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:04:13.660403 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ecc2f-6cb9-4b4b-966e-bfa882c9875c None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:13.701542 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:13.715057 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 539/1090] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:13.728670 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:13.744749 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9604d72-1bfe-4a47-93d3-6f4224cdd3e5 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:13.783456 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ecc2f-6cb9-4b4b-966e-bfa882c9875c None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:13.826294 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b87dc2-cc7f-476f-9b4a-0ed0ac41942a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:13.827115 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b87dc2-cc7f-476f-9b4a-0ed0ac41942a None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:13.827404 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b87dc2-cc7f-476f-9b4a-0ed0ac41942a None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:13.827682 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b87dc2-cc7f-476f-9b4a-0ed0ac41942a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:13.839197 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b87dc2-cc7f-476f-9b4a-0ed0ac41942a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:13.840773 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b87dc2-cc7f-476f-9b4a-0ed0ac41942a None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:13.853226 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9604d72-1bfe-4a47-93d3-6f4224cdd3e5 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:13.861152 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-e7b87dc2-cc7f-476f-9b4a-0ed0ac41942a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:13.861152 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:13.861152 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:13.861152 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:13.861152 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:13.861152 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:13.861152 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:13.861152 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:13.861152 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:13.861152 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:13.861152 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:13.861152 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:13.861152 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:04:13.861152 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:04:13.861152 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:04:13.862172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:04:13.862172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:13.862172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:13.862172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:13.862172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:13.862172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:13.862172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:13.862172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:13.862172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:13.862172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:13.862172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:13.862172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:13.862172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:13.862172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:13.862172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:13.862172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:13.862172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:13.863182 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:04:13.863182 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:04:13.863182 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:13.863182 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:04:13.863182 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 552/1091] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:13.868284 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b87dc2-cc7f-476f-9b4a-0ed0ac41942a None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:13.891215 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b10541-2aca-4a14-a7c3-0ff84d6758d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:13.891876 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b10541-2aca-4a14-a7c3-0ff84d6758d6 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:13.892159 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b10541-2aca-4a14-a7c3-0ff84d6758d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:13.892435 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b10541-2aca-4a14-a7c3-0ff84d6758d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:13.957545 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:13.971259 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b10541-2aca-4a14-a7c3-0ff84d6758d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:13.972890 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b10541-2aca-4a14-a7c3-0ff84d6758d6 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:13.977936 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b87dc2-cc7f-476f-9b4a-0ed0ac41942a None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:13.990087 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:14.021471 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfa8af3-ddca-437c-a9a3-f39d6165bf55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:14.022047 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa8af3-ddca-437c-a9a3-f39d6165bf55 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:14.022351 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa8af3-ddca-437c-a9a3-f39d6165bf55 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:14.022610 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa8af3-ddca-437c-a9a3-f39d6165bf55 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:14.023342 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfa8af3-ddca-437c-a9a3-f39d6165bf55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:14.024102 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfa8af3-ddca-437c-a9a3-f39d6165bf55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:14.025563 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfa8af3-ddca-437c-a9a3-f39d6165bf55 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:14.039161 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 553/1092] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:04:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:04:14.044530 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfa8af3-ddca-437c-a9a3-f39d6165bf55 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:14.089906 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 540/1093] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:04:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 27 21:04:14.095825 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b10541-2aca-4a14-a7c3-0ff84d6758d6 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:14.148225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfa8af3-ddca-437c-a9a3-f39d6165bf55 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:14.183649 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c3ce12-add3-4bba-9737-0ff43351f7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:14.184541 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3ce12-add3-4bba-9737-0ff43351f7d2 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:14.184890 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3ce12-add3-4bba-9737-0ff43351f7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:14.185255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c3ce12-add3-4bba-9737-0ff43351f7d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:14.194665 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c3ce12-add3-4bba-9737-0ff43351f7d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:14.196140 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c3ce12-add3-4bba-9737-0ff43351f7d2 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:14.203753 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:14.210786 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-61c3ce12-add3-4bba-9737-0ff43351f7d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:04:14.210786 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:14.210786 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:14.210786 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:14.210786 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:14.210786 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:14.210786 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:14.210786 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:14.210786 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:14.210786 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:14.210786 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:14.210786 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:14.210786 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:04:14.210786 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:04:14.210786 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:04:14.211879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:04:14.211879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:14.211879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:14.211879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:14.211879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:14.211879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:14.211879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:14.211879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:14.211879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:14.211879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:14.211879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:14.211879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:14.211879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:14.211879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:14.211879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:14.211879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:14.211879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:14.213998 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:04:14.213998 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:04:14.213998 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:04:14.213998 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:04:14.213998 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:04:14.213998 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:04:14.213998 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:04:14.213998 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:04:14.213998 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 554/1094] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:04:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:14.217081 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b10541-2aca-4a14-a7c3-0ff84d6758d6 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:14.218140 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c3ce12-add3-4bba-9737-0ff43351f7d2 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:14.252352 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:14.255686 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de85373d-ba59-4e99-8447-8f1c5402b1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:14.256523 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85373d-ba59-4e99-8447-8f1c5402b1ae None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:14.256812 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85373d-ba59-4e99-8447-8f1c5402b1ae None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:14.257117 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85373d-ba59-4e99-8447-8f1c5402b1ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:14.270254 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de85373d-ba59-4e99-8447-8f1c5402b1ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:14.271732 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de85373d-ba59-4e99-8447-8f1c5402b1ae None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:14.290037 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-de85373d-ba59-4e99-8447-8f1c5402b1ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:14.290037 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:14.290037 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:14.290037 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:14.290037 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:14.290037 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:14.290037 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:14.290037 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:14.290037 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:14.290037 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:14.290037 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:14.290037 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:14.290037 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:04:14.290037 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:04:14.290037 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:04:14.291056 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:04:14.291056 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:14.291056 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:14.291056 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:14.291056 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:14.291056 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:14.291056 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:14.291056 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:14.291056 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:14.291056 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:14.291056 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:14.291056 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:14.291056 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:14.291056 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:14.291056 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:14.291056 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:14.291056 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:14.292167 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:04:14.292167 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:04:14.292167 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:04:14.292167 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:04:14.292167 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 541/1095] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:04:14.297163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de85373d-ba59-4e99-8447-8f1c5402b1ae None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:14.357883 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c3ce12-add3-4bba-9737-0ff43351f7d2 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:14.392202 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cd0b69-5536-45f1-a130-d819cbc2397c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:14.393066 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd0b69-5536-45f1-a130-d819cbc2397c None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:14.393350 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd0b69-5536-45f1-a130-d819cbc2397c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:14.393912 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cd0b69-5536-45f1-a130-d819cbc2397c None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:14.394709 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cd0b69-5536-45f1-a130-d819cbc2397c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:14.395018 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cd0b69-5536-45f1-a130-d819cbc2397c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:14.396878 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cd0b69-5536-45f1-a130-d819cbc2397c None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:14.409421 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 555/1096] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 27 21:04:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:14.410059 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de85373d-ba59-4e99-8447-8f1c5402b1ae None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:14.415132 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cd0b69-5536-45f1-a130-d819cbc2397c None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:14.447532 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6dfaa1-b4d0-477c-8f78-6ff36fb420b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:14.448201 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6dfaa1-b4d0-477c-8f78-6ff36fb420b3 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:14.448482 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6dfaa1-b4d0-477c-8f78-6ff36fb420b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:14.448781 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6dfaa1-b4d0-477c-8f78-6ff36fb420b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:14.449511 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6dfaa1-b4d0-477c-8f78-6ff36fb420b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:14.449815 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6dfaa1-b4d0-477c-8f78-6ff36fb420b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:14.451568 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6dfaa1-b4d0-477c-8f78-6ff36fb420b3 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:14.463891 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 542/1097] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:04:14.513367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:14.524864 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cd0b69-5536-45f1-a130-d819cbc2397c None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:14.566333 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b83ac0-179f-416f-a08f-125c580537bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:14.567008 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b83ac0-179f-416f-a08f-125c580537bb None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:14.567286 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b83ac0-179f-416f-a08f-125c580537bb None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:14.567556 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b83ac0-179f-416f-a08f-125c580537bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:14.568408 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b83ac0-179f-416f-a08f-125c580537bb None admin] RBAC: Adding query filter params (role.id=b3dcc5de8383479d995a9bc38d8a8ef2, scope.project.id=334438c929c44e489e9a3b93e6d37231, user.id=1366958034d540b4ba2feb421a6afabc) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:14.568716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b83ac0-179f-416f-a08f-125c580537bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:14.570227 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b83ac0-179f-416f-a08f-125c580537bb None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:14.584983 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 556/1098] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:04:14 2022] GET /identity/v3/role_assignments?user.id=1366958034d540b4ba2feb421a6afabc&scope.project.id=334438c929c44e489e9a3b93e6d37231&role.id=b3dcc5de8383479d995a9bc38d8a8ef2 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:04:15.690547 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384926b-13e9-4549-aa1d-18d2c34e7a1d None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:15.691457 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384926b-13e9-4549-aa1d-18d2c34e7a1d None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:15.691457 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384926b-13e9-4549-aa1d-18d2c34e7a1d None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:15.693675 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 543/1099] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:04:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:15.700016 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ab361-7763-4232-a087-8ad0175425b7 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:15.700484 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ab361-7763-4232-a087-8ad0175425b7 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:15.700594 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ab361-7763-4232-a087-8ad0175425b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:15.723233 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-cb6ab361-7763-4232-a087-8ad0175425b7 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:15.724431 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-cb6ab361-7763-4232-a087-8ad0175425b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:15.799079 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 557/1100] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:15.823228 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2384926b-13e9-4549-aa1d-18d2c34e7a1d None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:15.827439 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:15.929493 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2384926b-13e9-4549-aa1d-18d2c34e7a1d None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:15.964229 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa3bfc0-a956-4706-b3c8-21e7b5fd324f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:15.964967 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa3bfc0-a956-4706-b3c8-21e7b5fd324f None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:15.965241 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa3bfc0-a956-4706-b3c8-21e7b5fd324f None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:15.965513 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa3bfc0-a956-4706-b3c8-21e7b5fd324f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:15.976864 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa3bfc0-a956-4706-b3c8-21e7b5fd324f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:04:15.984362 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-1fa3bfc0-a956-4706-b3c8-21e7b5fd324f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:04:15.984362 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:15.984362 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:15.984362 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:15.984362 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:15.984362 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:15.984362 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:15.984362 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:15.984362 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:15.984362 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:15.984362 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:15.984362 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:15.984362 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:04:15.984362 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:04:15.984362 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:04:15.984362 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:04:15.986138 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:04:15.986138 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:04:15.986138 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:04:15.986138 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:04:15.986138 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:15.986138 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:15.986138 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:15.986138 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:15.986138 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:15.986138 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:15.986138 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:15.986138 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:15.986138 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:15.986138 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:15.986138 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:15.986138 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:15.987304 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:15.987304 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:15.987304 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:15.987304 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:15.987304 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:04:15.987304 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:04:15.987304 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:04:15.987304 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:04:15.987304 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:04:15.987304 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:04:15.987304 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:04:15.987304 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:04:15.987304 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 544/1101] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 27 21:04:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:15.991857 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ab361-7763-4232-a087-8ad0175425b7 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:16.088290 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:16.109537 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ab361-7763-4232-a087-8ad0175425b7 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:16.147285 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a847916f-730f-4e7e-848b-3a60d4896f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:16.147952 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a847916f-730f-4e7e-848b-3a60d4896f34 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:16.148231 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a847916f-730f-4e7e-848b-3a60d4896f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:16.148501 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a847916f-730f-4e7e-848b-3a60d4896f34 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:16.149280 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a847916f-730f-4e7e-848b-3a60d4896f34 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:16.149581 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a847916f-730f-4e7e-848b-3a60d4896f34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:16.151382 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a847916f-730f-4e7e-848b-3a60d4896f34 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:16.159755 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 558/1102] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:04:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:16.166606 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa3bfc0-a956-4706-b3c8-21e7b5fd324f None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:16.237459 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:16.276432 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa3bfc0-a956-4706-b3c8-21e7b5fd324f None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:16.314317 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1261f79-94dd-4ee0-b2cb-5a6228e75af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:16.315230 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1261f79-94dd-4ee0-b2cb-5a6228e75af4 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:16.315468 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1261f79-94dd-4ee0-b2cb-5a6228e75af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:16.315739 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1261f79-94dd-4ee0-b2cb-5a6228e75af4 None admin] PATH_INFO: `/v3/users/87f38a9c73404af7b8cff77ab652599a` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:16.316962 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1261f79-94dd-4ee0-b2cb-5a6228e75af4 None admin] RBAC: Authorizing `identity:get_user(user_id=87f38a9c73404af7b8cff77ab652599a)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:16.318419 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1261f79-94dd-4ee0-b2cb-5a6228e75af4 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:16.320095 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 545/1103] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:04:16 2022] GET /identity/v3/users/87f38a9c73404af7b8cff77ab652599a => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:16.331112 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a847916f-730f-4e7e-848b-3a60d4896f34 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:16.447020 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a847916f-730f-4e7e-848b-3a60d4896f34 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:16.483671 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aac840-cbe9-46e6-a791-2ee020b79644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:16.484324 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aac840-cbe9-46e6-a791-2ee020b79644 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:16.484605 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aac840-cbe9-46e6-a791-2ee020b79644 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:16.484880 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aac840-cbe9-46e6-a791-2ee020b79644 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:16.504331 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:16.567343 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aac840-cbe9-46e6-a791-2ee020b79644 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:16.569596 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aac840-cbe9-46e6-a791-2ee020b79644 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:16.680324 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 559/1104] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:04:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 21:04:16.688477 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1261f79-94dd-4ee0-b2cb-5a6228e75af4 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:16.798867 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1261f79-94dd-4ee0-b2cb-5a6228e75af4 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:16.832386 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:16.839514 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94340e0-8c16-4bfa-b613-90da8991f170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:16.840386 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94340e0-8c16-4bfa-b613-90da8991f170 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:16.840664 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94340e0-8c16-4bfa-b613-90da8991f170 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:16.840956 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94340e0-8c16-4bfa-b613-90da8991f170 None admin] PATH_INFO: `/v3/projects/0fcb2a9eb1314203b57d999c396a0f9d` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:16.842137 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94340e0-8c16-4bfa-b613-90da8991f170 None admin] RBAC: Authorizing `identity:get_project(project_id=0fcb2a9eb1314203b57d999c396a0f9d)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:16.843575 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94340e0-8c16-4bfa-b613-90da8991f170 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:16.845353 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 546/1105] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:04:16 2022] GET /identity/v3/projects/0fcb2a9eb1314203b57d999c396a0f9d => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:16.853871 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aac840-cbe9-46e6-a791-2ee020b79644 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:16.909162 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:16.965815 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aac840-cbe9-46e6-a791-2ee020b79644 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:17.003412 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c72f4-88a6-424b-a8b1-1f3d7f96fa23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:17.004093 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c72f4-88a6-424b-a8b1-1f3d7f96fa23 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:17.004341 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c72f4-88a6-424b-a8b1-1f3d7f96fa23 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:17.004614 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c72f4-88a6-424b-a8b1-1f3d7f96fa23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:17.005457 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c72f4-88a6-424b-a8b1-1f3d7f96fa23 None admin] RBAC: Adding query filter params (role.id=d336dd7b746c4488b33289c00302e4d5, scope.project.id=0fcb2a9eb1314203b57d999c396a0f9d, user.id=87f38a9c73404af7b8cff77ab652599a) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:17.005766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c72f4-88a6-424b-a8b1-1f3d7f96fa23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:17.007207 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c72f4-88a6-424b-a8b1-1f3d7f96fa23 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:17.013825 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 560/1106] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:04:16 2022] GET /identity/v3/role_assignments?user.id=87f38a9c73404af7b8cff77ab652599a&scope.project.id=0fcb2a9eb1314203b57d999c396a0f9d&role.id=d336dd7b746c4488b33289c00302e4d5 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:19.090295 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04448bbe-7fac-435d-b222-c90e30081a53 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:19.090901 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04448bbe-7fac-435d-b222-c90e30081a53 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:19.090901 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04448bbe-7fac-435d-b222-c90e30081a53 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:19.092167 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 547/1107] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:04:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:19.098129 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ed08b-5d95-47b6-aae5-6081edc4b5c2 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:19.098580 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ed08b-5d95-47b6-aae5-6081edc4b5c2 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:19.098675 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ed08b-5d95-47b6-aae5-6081edc4b5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:19.116852 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-5a9ed08b-5d95-47b6-aae5-6081edc4b5c2 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:19.118089 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-5a9ed08b-5d95-47b6-aae5-6081edc4b5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:19.198445 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 561/1108] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:19.223907 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04448bbe-7fac-435d-b222-c90e30081a53 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:19.336871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04448bbe-7fac-435d-b222-c90e30081a53 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:19.378336 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3316d522-919b-4971-b718-23cf4f4ed558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:19.379137 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316d522-919b-4971-b718-23cf4f4ed558 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:19.379400 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316d522-919b-4971-b718-23cf4f4ed558 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:19.379676 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316d522-919b-4971-b718-23cf4f4ed558 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:19.380875 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3316d522-919b-4971-b718-23cf4f4ed558 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:19.382346 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3316d522-919b-4971-b718-23cf4f4ed558 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:19.383961 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 548/1109] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:19.390702 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ed08b-5d95-47b6-aae5-6081edc4b5c2 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:19.436298 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:19.500100 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ed08b-5d95-47b6-aae5-6081edc4b5c2 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:19.542938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e933a-377a-462d-9782-4336cffb01b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:19.543616 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e933a-377a-462d-9782-4336cffb01b2 None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:19.543897 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e933a-377a-462d-9782-4336cffb01b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:19.544157 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e933a-377a-462d-9782-4336cffb01b2 None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:19.544882 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61e933a-377a-462d-9782-4336cffb01b2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:19.546375 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61e933a-377a-462d-9782-4336cffb01b2 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:19.568824 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 562/1110] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Jun 27 21:04:19 2022] POST /identity/v3/users => generated 286 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:04:21.672963 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4dad0-40d7-43fa-a7e3-d6070b1bb254 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:21.673530 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4dad0-40d7-43fa-a7e3-d6070b1bb254 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:21.673530 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4dad0-40d7-43fa-a7e3-d6070b1bb254 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:21.674920 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 549/1111] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:21.680390 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c448f-5219-4090-b267-f7485cd4003f None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:21.680846 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c448f-5219-4090-b267-f7485cd4003f None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:21.680971 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c448f-5219-4090-b267-f7485cd4003f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:21.699009 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-898c448f-5219-4090-b267-f7485cd4003f None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:21.700227 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-898c448f-5219-4090-b267-f7485cd4003f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:21.774999 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 563/1112] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:21.798889 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f4dad0-40d7-43fa-a7e3-d6070b1bb254 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:21.877606 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:21.914563 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f4dad0-40d7-43fa-a7e3-d6070b1bb254 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:21.952494 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c195654-e6e5-40bc-ab7d-54fa19d9312e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:21.953297 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c195654-e6e5-40bc-ab7d-54fa19d9312e None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:21.953569 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c195654-e6e5-40bc-ab7d-54fa19d9312e None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:21.953923 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c195654-e6e5-40bc-ab7d-54fa19d9312e None admin] PATH_INFO: `/v3/roles/5834784bf54d4806b88e32e0fb8e18fe` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:21.955530 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c195654-e6e5-40bc-ab7d-54fa19d9312e None admin] RBAC: Authorizing `identity:get_role(role_id=5834784bf54d4806b88e32e0fb8e18fe)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:21.956898 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c195654-e6e5-40bc-ab7d-54fa19d9312e None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:21.958324 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 550/1113] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:04:21 2022] GET /identity/v3/roles/5834784bf54d4806b88e32e0fb8e18fe => generated 223 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:21.965034 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c448f-5219-4090-b267-f7485cd4003f None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:21.995239 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:22.074557 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c448f-5219-4090-b267-f7485cd4003f None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:22.111505 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74b7fbe-4bfc-480f-b5fd-a5285e1680ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:22.112385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b7fbe-4bfc-480f-b5fd-a5285e1680ff None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:22.112664 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b7fbe-4bfc-480f-b5fd-a5285e1680ff None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:22.112961 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b7fbe-4bfc-480f-b5fd-a5285e1680ff None admin] PATH_INFO: `/v3/users/681eee868a384c70be40273526bb81a5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:22.128314 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74b7fbe-4bfc-480f-b5fd-a5285e1680ff None admin] RBAC: Authorizing `identity:get_user(user_id=681eee868a384c70be40273526bb81a5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:22.129801 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74b7fbe-4bfc-480f-b5fd-a5285e1680ff None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:22.131552 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 564/1114] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:04:21 2022] GET /identity/v3/users/681eee868a384c70be40273526bb81a5 => generated 286 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:22.139358 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c195654-e6e5-40bc-ab7d-54fa19d9312e None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:22.252295 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c195654-e6e5-40bc-ab7d-54fa19d9312e None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:22.290005 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d7317-39d4-4490-8b43-cc86ed07698e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:22.290662 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d7317-39d4-4490-8b43-cc86ed07698e None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:22.291256 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d7317-39d4-4490-8b43-cc86ed07698e None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:22.291535 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818d7317-39d4-4490-8b43-cc86ed07698e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:22.295297 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:22.363969 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818d7317-39d4-4490-8b43-cc86ed07698e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:22.365465 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818d7317-39d4-4490-8b43-cc86ed07698e None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:22.471431 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 551/1115] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:04:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 21:04:22.478077 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74b7fbe-4bfc-480f-b5fd-a5285e1680ff None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:22.567028 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:22.584777 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74b7fbe-4bfc-480f-b5fd-a5285e1680ff None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:22.623207 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09573297-c7e9-4d07-ad08-13d59c47a18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:22.624079 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09573297-c7e9-4d07-ad08-13d59c47a18d None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:22.624357 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09573297-c7e9-4d07-ad08-13d59c47a18d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:22.624624 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09573297-c7e9-4d07-ad08-13d59c47a18d None admin] PATH_INFO: `/v3/projects/0fcb2a9eb1314203b57d999c396a0f9d` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:22.629423 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09573297-c7e9-4d07-ad08-13d59c47a18d None admin] RBAC: Authorizing `identity:get_project(project_id=0fcb2a9eb1314203b57d999c396a0f9d)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:22.630885 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09573297-c7e9-4d07-ad08-13d59c47a18d None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:22.632647 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 565/1116] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:04:22 2022] GET /identity/v3/projects/0fcb2a9eb1314203b57d999c396a0f9d => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:22.639271 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d7317-39d4-4490-8b43-cc86ed07698e None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:22.678816 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:22.755777 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818d7317-39d4-4490-8b43-cc86ed07698e None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:22.791674 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850eeb60-5422-4cb5-b50b-3a38d86e5238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:22.792349 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850eeb60-5422-4cb5-b50b-3a38d86e5238 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:22.792616 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850eeb60-5422-4cb5-b50b-3a38d86e5238 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:22.792891 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850eeb60-5422-4cb5-b50b-3a38d86e5238 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:22.793759 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850eeb60-5422-4cb5-b50b-3a38d86e5238 None admin] RBAC: Adding query filter params (role.id=5834784bf54d4806b88e32e0fb8e18fe, scope.project.id=0fcb2a9eb1314203b57d999c396a0f9d, user.id=681eee868a384c70be40273526bb81a5) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:22.794071 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850eeb60-5422-4cb5-b50b-3a38d86e5238 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:22.795517 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850eeb60-5422-4cb5-b50b-3a38d86e5238 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:22.801409 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 552/1117] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:04:22 2022] GET /identity/v3/role_assignments?user.id=681eee868a384c70be40273526bb81a5&scope.project.id=0fcb2a9eb1314203b57d999c396a0f9d&role.id=5834784bf54d4806b88e32e0fb8e18fe => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:24.881686 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36de3a-d28b-42c9-8a33-b8ddc0434dcb None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:24.882297 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36de3a-d28b-42c9-8a33-b8ddc0434dcb None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:24.882297 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36de3a-d28b-42c9-8a33-b8ddc0434dcb None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:24.883618 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 566/1118] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:04:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:24.890511 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65529525-a96a-4720-a578-02d6df867232 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:24.891043 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65529525-a96a-4720-a578-02d6df867232 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:24.891154 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65529525-a96a-4720-a578-02d6df867232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:24.914928 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-65529525-a96a-4720-a578-02d6df867232 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:24.916472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-65529525-a96a-4720-a578-02d6df867232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:25.001411 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 553/1119] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:25.024849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36de3a-d28b-42c9-8a33-b8ddc0434dcb None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:25.144046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36de3a-d28b-42c9-8a33-b8ddc0434dcb None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:25.188121 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264605c-da5b-4b70-9596-1b00d62a3283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:25.188973 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264605c-da5b-4b70-9596-1b00d62a3283 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:25.189272 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264605c-da5b-4b70-9596-1b00d62a3283 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:25.189542 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264605c-da5b-4b70-9596-1b00d62a3283 None admin] PATH_INFO: `/v3/roles/5834784bf54d4806b88e32e0fb8e18fe` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:25.191064 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f264605c-da5b-4b70-9596-1b00d62a3283 None admin] RBAC: Authorizing `identity:get_role(role_id=5834784bf54d4806b88e32e0fb8e18fe)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:25.192444 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f264605c-da5b-4b70-9596-1b00d62a3283 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:25.193870 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 567/1120] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:04:25 2022] GET /identity/v3/roles/5834784bf54d4806b88e32e0fb8e18fe => generated 223 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:25.200462 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65529525-a96a-4720-a578-02d6df867232 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:25.234775 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:25.314038 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65529525-a96a-4720-a578-02d6df867232 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:25.351669 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95629496-4d50-4196-b350-f1227b256d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:25.352557 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95629496-4d50-4196-b350-f1227b256d15 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:25.352835 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95629496-4d50-4196-b350-f1227b256d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:25.353159 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95629496-4d50-4196-b350-f1227b256d15 None admin] PATH_INFO: `/v3/users/681eee868a384c70be40273526bb81a5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:25.354601 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95629496-4d50-4196-b350-f1227b256d15 None admin] RBAC: Authorizing `identity:get_user(user_id=681eee868a384c70be40273526bb81a5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:25.356050 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95629496-4d50-4196-b350-f1227b256d15 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:25.357760 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 554/1121] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:04:25 2022] GET /identity/v3/users/681eee868a384c70be40273526bb81a5 => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:25.364009 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264605c-da5b-4b70-9596-1b00d62a3283 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:25.388430 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:25.472530 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264605c-da5b-4b70-9596-1b00d62a3283 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:25.509539 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac893b2-1b0a-4b97-8546-5898c83ea2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:25.510494 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac893b2-1b0a-4b97-8546-5898c83ea2a5 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:25.510779 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac893b2-1b0a-4b97-8546-5898c83ea2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:25.511062 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac893b2-1b0a-4b97-8546-5898c83ea2a5 None admin] PATH_INFO: `/v3/projects/0fcb2a9eb1314203b57d999c396a0f9d` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:25.512301 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac893b2-1b0a-4b97-8546-5898c83ea2a5 None admin] RBAC: Authorizing `identity:get_project(project_id=0fcb2a9eb1314203b57d999c396a0f9d)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:25.513790 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac893b2-1b0a-4b97-8546-5898c83ea2a5 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:25.515552 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 568/1122] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:04:25 2022] GET /identity/v3/projects/0fcb2a9eb1314203b57d999c396a0f9d => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:25.521831 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95629496-4d50-4196-b350-f1227b256d15 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:25.631500 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:25.637325 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95629496-4d50-4196-b350-f1227b256d15 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:25.674604 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79944895-a6cb-4bfc-89b6-316d24bf7794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:25.675486 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79944895-a6cb-4bfc-89b6-316d24bf7794 None admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:25.675766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79944895-a6cb-4bfc-89b6-316d24bf7794 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:25.676038 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79944895-a6cb-4bfc-89b6-316d24bf7794 None admin] PATH_INFO: `/v3/projects/0fcb2a9eb1314203b57d999c396a0f9d/users/681eee868a384c70be40273526bb81a5/roles/5834784bf54d4806b88e32e0fb8e18fe` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:25.678559 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79944895-a6cb-4bfc-89b6-316d24bf7794 None admin] RBAC: Authorizing `identity:create_grant(project_id=0fcb2a9eb1314203b57d999c396a0f9d, user_id=681eee868a384c70be40273526bb81a5, role_id=5834784bf54d4806b88e32e0fb8e18fe)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:25.680009 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79944895-a6cb-4bfc-89b6-316d24bf7794 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:25.694552 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 555/1123] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 27 21:04:25 2022] PUT /identity/v3/projects/0fcb2a9eb1314203b57d999c396a0f9d/users/681eee868a384c70be40273526bb81a5/roles/5834784bf54d4806b88e32e0fb8e18fe => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:04:27.739110 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d159ee-adb1-4066-a901-c2a8d524420d None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:27.739689 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d159ee-adb1-4066-a901-c2a8d524420d None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:27.739999 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d159ee-adb1-4066-a901-c2a8d524420d None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:27.742383 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 569/1124] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:04:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:27.748298 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd35f5-42f4-44cb-aaa4-d68bc8d34c05 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:27.748817 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd35f5-42f4-44cb-aaa4-d68bc8d34c05 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:27.748817 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd35f5-42f4-44cb-aaa4-d68bc8d34c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:27.767476 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-5bbd35f5-42f4-44cb-aaa4-d68bc8d34c05 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:27.768647 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-5bbd35f5-42f4-44cb-aaa4-d68bc8d34c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:27.838615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 556/1125] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:27.862917 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d159ee-adb1-4066-a901-c2a8d524420d None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:27.962123 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:27.977763 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d159ee-adb1-4066-a901-c2a8d524420d None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:28.020949 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80655a2-bb7e-4461-aae3-072d13f4e6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:28.021822 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80655a2-bb7e-4461-aae3-072d13f4e6bd None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:28.022102 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80655a2-bb7e-4461-aae3-072d13f4e6bd None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:28.022374 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80655a2-bb7e-4461-aae3-072d13f4e6bd None admin] PATH_INFO: `/v3/roles/5834784bf54d4806b88e32e0fb8e18fe` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:28.023941 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80655a2-bb7e-4461-aae3-072d13f4e6bd None admin] RBAC: Authorizing `identity:get_role(role_id=5834784bf54d4806b88e32e0fb8e18fe)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:28.025427 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80655a2-bb7e-4461-aae3-072d13f4e6bd None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:28.026831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 570/1126] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:04:27 2022] GET /identity/v3/roles/5834784bf54d4806b88e32e0fb8e18fe => generated 223 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:28.032868 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd35f5-42f4-44cb-aaa4-d68bc8d34c05 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:28.125295 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:28.138864 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd35f5-42f4-44cb-aaa4-d68bc8d34c05 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:28.175914 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8256ab-b389-49ff-8cbb-2dfc21d44c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:28.176833 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8256ab-b389-49ff-8cbb-2dfc21d44c5b None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:28.177149 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8256ab-b389-49ff-8cbb-2dfc21d44c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:28.177407 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8256ab-b389-49ff-8cbb-2dfc21d44c5b None admin] PATH_INFO: `/v3/users/681eee868a384c70be40273526bb81a5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:28.178639 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8256ab-b389-49ff-8cbb-2dfc21d44c5b None admin] RBAC: Authorizing `identity:get_user(user_id=681eee868a384c70be40273526bb81a5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:28.180090 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8256ab-b389-49ff-8cbb-2dfc21d44c5b None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:28.181978 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 557/1127] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:04:28 2022] GET /identity/v3/users/681eee868a384c70be40273526bb81a5 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:28.188016 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80655a2-bb7e-4461-aae3-072d13f4e6bd None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:28.308301 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80655a2-bb7e-4461-aae3-072d13f4e6bd None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:28.343691 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee718ad0-a290-4e5a-968f-66ecfcbf7c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:28.344332 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee718ad0-a290-4e5a-968f-66ecfcbf7c6f None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:28.344609 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee718ad0-a290-4e5a-968f-66ecfcbf7c6f None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:28.344878 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee718ad0-a290-4e5a-968f-66ecfcbf7c6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:28.372114 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:28.417951 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee718ad0-a290-4e5a-968f-66ecfcbf7c6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:28.419400 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee718ad0-a290-4e5a-968f-66ecfcbf7c6f None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:28.539659 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 571/1128] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:04:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 21:04:28.546048 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8256ab-b389-49ff-8cbb-2dfc21d44c5b None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:28.661627 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8256ab-b389-49ff-8cbb-2dfc21d44c5b None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:28.697534 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1865b13-dfa6-481e-8d4f-9f2025b1b0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:28.698578 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1865b13-dfa6-481e-8d4f-9f2025b1b0fe None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:28.698859 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1865b13-dfa6-481e-8d4f-9f2025b1b0fe None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:28.699132 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1865b13-dfa6-481e-8d4f-9f2025b1b0fe None admin] PATH_INFO: `/v3/projects/0fcb2a9eb1314203b57d999c396a0f9d` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:28.700388 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1865b13-dfa6-481e-8d4f-9f2025b1b0fe None admin] RBAC: Authorizing `identity:get_project(project_id=0fcb2a9eb1314203b57d999c396a0f9d)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:28.701840 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1865b13-dfa6-481e-8d4f-9f2025b1b0fe None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:28.703608 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 558/1129] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:04:28 2022] GET /identity/v3/projects/0fcb2a9eb1314203b57d999c396a0f9d => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:28.710686 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee718ad0-a290-4e5a-968f-66ecfcbf7c6f None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:28.771522 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:28.816724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee718ad0-a290-4e5a-968f-66ecfcbf7c6f None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:28.853476 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93678a1-f6ef-4824-b2cf-aa4bfcc30c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:28.854130 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93678a1-f6ef-4824-b2cf-aa4bfcc30c8c None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:28.854416 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93678a1-f6ef-4824-b2cf-aa4bfcc30c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:28.854688 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93678a1-f6ef-4824-b2cf-aa4bfcc30c8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:28.855530 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93678a1-f6ef-4824-b2cf-aa4bfcc30c8c None admin] RBAC: Adding query filter params (role.id=5834784bf54d4806b88e32e0fb8e18fe, scope.project.id=0fcb2a9eb1314203b57d999c396a0f9d, user.id=681eee868a384c70be40273526bb81a5) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:28.855842 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93678a1-f6ef-4824-b2cf-aa4bfcc30c8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:28.857324 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93678a1-f6ef-4824-b2cf-aa4bfcc30c8c None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:28.863888 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 572/1130] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:04:28 2022] GET /identity/v3/role_assignments?user.id=681eee868a384c70be40273526bb81a5&scope.project.id=0fcb2a9eb1314203b57d999c396a0f9d&role.id=5834784bf54d4806b88e32e0fb8e18fe => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:30.940962 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66c48a-9130-4963-a974-b1ef768c6197 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:30.941541 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66c48a-9130-4963-a974-b1ef768c6197 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:30.941541 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf66c48a-9130-4963-a974-b1ef768c6197 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:30.942850 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 559/1131] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:04:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:30.949138 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660dd16b-6405-4fa2-a8be-30f032353e41 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:30.949687 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660dd16b-6405-4fa2-a8be-30f032353e41 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:30.949687 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660dd16b-6405-4fa2-a8be-30f032353e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:30.968515 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-660dd16b-6405-4fa2-a8be-30f032353e41 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:30.969786 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-660dd16b-6405-4fa2-a8be-30f032353e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:31.057286 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 573/1132] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:31.080955 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66c48a-9130-4963-a974-b1ef768c6197 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:31.109141 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:31.198644 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf66c48a-9130-4963-a974-b1ef768c6197 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:31.235933 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9761f9-67c7-4019-93a9-a6376abe948c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:31.236729 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9761f9-67c7-4019-93a9-a6376abe948c None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:31.237000 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9761f9-67c7-4019-93a9-a6376abe948c None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:31.237303 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9761f9-67c7-4019-93a9-a6376abe948c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:31.238466 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9761f9-67c7-4019-93a9-a6376abe948c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:31.239915 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9761f9-67c7-4019-93a9-a6376abe948c None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:31.241568 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 560/1133] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:31.248421 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660dd16b-6405-4fa2-a8be-30f032353e41 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:31.301748 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:31.362877 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660dd16b-6405-4fa2-a8be-30f032353e41 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:31.401923 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3c728-b513-4ae7-80ad-e42093afee06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:31.403040 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3c728-b513-4ae7-80ad-e42093afee06 None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:31.403040 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3c728-b513-4ae7-80ad-e42093afee06 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:31.403227 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3c728-b513-4ae7-80ad-e42093afee06 None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:31.403837 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd3c728-b513-4ae7-80ad-e42093afee06 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:31.405329 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd3c728-b513-4ae7-80ad-e42093afee06 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:31.432474 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 574/1134] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Jun 27 21:04:31 2022] POST /identity/v3/projects => generated 311 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:04:33.514053 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c10b0-3310-4047-8c8f-2157dbe6843e None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:33.514655 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c10b0-3310-4047-8c8f-2157dbe6843e None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:33.514655 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c10b0-3310-4047-8c8f-2157dbe6843e None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:33.515953 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 561/1135] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:04:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:33.521704 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401744b0-60b5-418d-a8ac-b11d7c1e29de None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:33.522170 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401744b0-60b5-418d-a8ac-b11d7c1e29de None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:33.522285 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401744b0-60b5-418d-a8ac-b11d7c1e29de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:33.540551 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-401744b0-60b5-418d-a8ac-b11d7c1e29de None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:33.541801 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-401744b0-60b5-418d-a8ac-b11d7c1e29de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:33.627685 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 575/1136] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:33.653812 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c10b0-3310-4047-8c8f-2157dbe6843e None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:33.769099 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:33.783985 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853c10b0-3310-4047-8c8f-2157dbe6843e None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:33.822241 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa15441-8e75-48ee-8d36-cc8373965e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:33.823111 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa15441-8e75-48ee-8d36-cc8373965e88 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:33.823394 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa15441-8e75-48ee-8d36-cc8373965e88 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:33.823760 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa15441-8e75-48ee-8d36-cc8373965e88 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:33.825298 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa15441-8e75-48ee-8d36-cc8373965e88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:33.826741 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa15441-8e75-48ee-8d36-cc8373965e88 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:33.828405 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 562/1137] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:04:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:33.834550 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401744b0-60b5-418d-a8ac-b11d7c1e29de None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:33.868839 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:33.946396 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401744b0-60b5-418d-a8ac-b11d7c1e29de None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:33.984728 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36226f56-fbe9-429e-a3f9-285ebd0452dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:33.985442 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36226f56-fbe9-429e-a3f9-285ebd0452dd None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:33.985717 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36226f56-fbe9-429e-a3f9-285ebd0452dd None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:33.985989 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36226f56-fbe9-429e-a3f9-285ebd0452dd None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:33.986713 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36226f56-fbe9-429e-a3f9-285ebd0452dd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:33.988150 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36226f56-fbe9-429e-a3f9-285ebd0452dd None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:34.012621 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 576/1138] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Jun 27 21:04:33 2022] POST /identity/v3/users => generated 286 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:04:36.134061 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22decfab-0e4d-45a7-a87d-1c235a2d272c None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:36.134627 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22decfab-0e4d-45a7-a87d-1c235a2d272c None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:36.134765 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22decfab-0e4d-45a7-a87d-1c235a2d272c None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:36.136682 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 563/1139] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:04:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:36.145734 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8d6e9-d4ad-438c-9c8d-7635198610d2 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:36.146522 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8d6e9-d4ad-438c-9c8d-7635198610d2 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:36.146648 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8d6e9-d4ad-438c-9c8d-7635198610d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:36.170268 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-fbb8d6e9-d4ad-438c-9c8d-7635198610d2 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:36.171441 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-fbb8d6e9-d4ad-438c-9c8d-7635198610d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:36.244539 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 577/1140] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:04:36.268992 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22decfab-0e4d-45a7-a87d-1c235a2d272c None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:36.379101 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22decfab-0e4d-45a7-a87d-1c235a2d272c None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:36.416506 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312c4f4-e09d-4dbe-b2cf-c49fe78ee9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:36.417358 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312c4f4-e09d-4dbe-b2cf-c49fe78ee9ea None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:36.417642 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312c4f4-e09d-4dbe-b2cf-c49fe78ee9ea None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:36.417919 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312c4f4-e09d-4dbe-b2cf-c49fe78ee9ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:36.429355 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0312c4f4-e09d-4dbe-b2cf-c49fe78ee9ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:04:36.437096 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-0312c4f4-e09d-4dbe-b2cf-c49fe78ee9ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:04:36.437096 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:36.437096 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:36.437096 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:36.437096 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:36.437096 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:36.437096 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:36.437096 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:36.437096 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:36.437096 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:36.437096 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:36.437096 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:36.437096 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:04:36.437096 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:04:36.437096 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:04:36.437096 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:04:36.438232 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:04:36.438232 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:04:36.438232 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:04:36.438232 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:04:36.438232 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:36.438232 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:36.438232 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:36.438232 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:36.438232 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:36.438232 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:36.438232 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:36.438232 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:36.438232 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:36.438232 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:36.438232 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:36.438232 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:36.439351 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:36.439351 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:36.439351 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:36.439351 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:36.439351 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:04:36.439351 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:04:36.439351 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:04:36.439351 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:04:36.439351 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:04:36.439351 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:04:36.439351 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:04:36.439351 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:04:36.439351 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 564/1141] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 27 21:04:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:36.444637 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb8d6e9-d4ad-438c-9c8d-7635198610d2 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:36.464011 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:36.552626 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb8d6e9-d4ad-438c-9c8d-7635198610d2 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:36.590866 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9062ddad-0669-4671-b815-eca7286fae84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:36.591540 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062ddad-0669-4671-b815-eca7286fae84 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:36.591820 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062ddad-0669-4671-b815-eca7286fae84 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:36.592098 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062ddad-0669-4671-b815-eca7286fae84 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:36.592854 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9062ddad-0669-4671-b815-eca7286fae84 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:36.593204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9062ddad-0669-4671-b815-eca7286fae84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:36.594613 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9062ddad-0669-4671-b815-eca7286fae84 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:36.604144 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 578/1142] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:04:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:36.610360 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312c4f4-e09d-4dbe-b2cf-c49fe78ee9ea None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:36.637771 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:36.739102 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312c4f4-e09d-4dbe-b2cf-c49fe78ee9ea None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:36.776394 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62868de8-190f-4ea7-a362-8bef56e763a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:36.777282 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62868de8-190f-4ea7-a362-8bef56e763a0 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:36.777582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62868de8-190f-4ea7-a362-8bef56e763a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:36.777826 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62868de8-190f-4ea7-a362-8bef56e763a0 None admin] PATH_INFO: `/v3/users/794f41c47bb34760af2f898710712869` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:36.799672 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62868de8-190f-4ea7-a362-8bef56e763a0 None admin] RBAC: Authorizing `identity:get_user(user_id=794f41c47bb34760af2f898710712869)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:36.801170 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62868de8-190f-4ea7-a362-8bef56e763a0 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:36.802980 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 565/1143] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:04:36 2022] GET /identity/v3/users/794f41c47bb34760af2f898710712869 => generated 286 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:36.809375 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9062ddad-0669-4671-b815-eca7286fae84 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:36.898638 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:36.919917 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9062ddad-0669-4671-b815-eca7286fae84 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:36.956957 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c40720-f913-451d-948f-001d5b3d3104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:36.957599 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c40720-f913-451d-948f-001d5b3d3104 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:36.957863 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c40720-f913-451d-948f-001d5b3d3104 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:36.958134 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c40720-f913-451d-948f-001d5b3d3104 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:37.031179 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c40720-f913-451d-948f-001d5b3d3104 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:37.032648 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c40720-f913-451d-948f-001d5b3d3104 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:37.117854 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:37.138744 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 579/1144] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:04:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 21:04:37.145037 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62868de8-190f-4ea7-a362-8bef56e763a0 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:37.231049 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:37.257580 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62868de8-190f-4ea7-a362-8bef56e763a0 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:37.295073 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a0321-b42e-49ca-adf2-9845bf055548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:37.295951 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a0321-b42e-49ca-adf2-9845bf055548 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:37.296232 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a0321-b42e-49ca-adf2-9845bf055548 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:37.296504 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a0321-b42e-49ca-adf2-9845bf055548 None admin] PATH_INFO: `/v3/projects/d2a30d1217f447ca911d699505efc965` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:37.297740 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64a0321-b42e-49ca-adf2-9845bf055548 None admin] RBAC: Authorizing `identity:get_project(project_id=d2a30d1217f447ca911d699505efc965)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:37.299175 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64a0321-b42e-49ca-adf2-9845bf055548 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:37.300907 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 566/1145] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:04:37 2022] GET /identity/v3/projects/d2a30d1217f447ca911d699505efc965 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:37.308380 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c40720-f913-451d-948f-001d5b3d3104 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:37.429813 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c40720-f913-451d-948f-001d5b3d3104 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:37.465355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d638b82-bb52-43af-8115-4e3226bdb6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:37.466005 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d638b82-bb52-43af-8115-4e3226bdb6ae None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:37.466250 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d638b82-bb52-43af-8115-4e3226bdb6ae None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:37.466519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d638b82-bb52-43af-8115-4e3226bdb6ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:37.467351 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d638b82-bb52-43af-8115-4e3226bdb6ae None admin] RBAC: Adding query filter params (role.id=d336dd7b746c4488b33289c00302e4d5, scope.project.id=d2a30d1217f447ca911d699505efc965, user.id=794f41c47bb34760af2f898710712869) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:37.467695 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d638b82-bb52-43af-8115-4e3226bdb6ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:37.469136 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d638b82-bb52-43af-8115-4e3226bdb6ae None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:37.474995 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 580/1146] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:04:37 2022] GET /identity/v3/role_assignments?user.id=794f41c47bb34760af2f898710712869&scope.project.id=d2a30d1217f447ca911d699505efc965&role.id=d336dd7b746c4488b33289c00302e4d5 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:39.503026 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c23ed-0728-41b8-9571-f758c59a4c37 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:39.503589 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c23ed-0728-41b8-9571-f758c59a4c37 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:39.503589 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c23ed-0728-41b8-9571-f758c59a4c37 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:39.505061 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 567/1147] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:04:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:39.510897 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b406f8-3e9f-4fd3-bfc6-49aead2695f2 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:39.511361 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b406f8-3e9f-4fd3-bfc6-49aead2695f2 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:39.511451 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b406f8-3e9f-4fd3-bfc6-49aead2695f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:39.529241 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-a5b406f8-3e9f-4fd3-bfc6-49aead2695f2 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:39.530472 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a5b406f8-3e9f-4fd3-bfc6-49aead2695f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:39.574989 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:39.607410 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 581/1148] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:39.630749 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69c23ed-0728-41b8-9571-f758c59a4c37 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:39.743249 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69c23ed-0728-41b8-9571-f758c59a4c37 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:39.779384 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b4fe7-df18-4fda-bc54-64640d44fcce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:39.780075 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b4fe7-df18-4fda-bc54-64640d44fcce None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:39.780354 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b4fe7-df18-4fda-bc54-64640d44fcce None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:39.780627 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b4fe7-df18-4fda-bc54-64640d44fcce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:39.791444 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b4fe7-df18-4fda-bc54-64640d44fcce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:04:39.799031 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-d79b4fe7-df18-4fda-bc54-64640d44fcce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:04:39.799031 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:39.799031 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:39.799031 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:39.799031 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:39.799031 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:39.799031 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:39.799031 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:39.799031 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:39.799031 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:39.799031 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:39.799031 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:39.799031 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:04:39.799031 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:04:39.799031 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:04:39.799031 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:04:39.800194 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:04:39.800194 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:04:39.800194 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:04:39.800194 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:04:39.800194 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:39.800194 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:39.800194 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:39.800194 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:39.800194 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:39.800194 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:39.800194 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:39.800194 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:39.800194 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:39.800194 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:39.800194 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:39.800194 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:39.801398 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:39.801398 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:39.801398 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:39.801398 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:39.801398 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:04:39.801398 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:04:39.801398 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:04:39.801398 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:04:39.801398 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:04:39.801398 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:04:39.801398 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:04:39.801398 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:04:39.801398 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 568/1149] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 27 21:04:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:39.810337 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b406f8-3e9f-4fd3-bfc6-49aead2695f2 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:39.921681 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b406f8-3e9f-4fd3-bfc6-49aead2695f2 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:39.957947 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd39045-f3d7-42fc-82cb-94ba8deee945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:39.958616 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd39045-f3d7-42fc-82cb-94ba8deee945 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:39.958897 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd39045-f3d7-42fc-82cb-94ba8deee945 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:39.959174 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd39045-f3d7-42fc-82cb-94ba8deee945 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:39.959928 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd39045-f3d7-42fc-82cb-94ba8deee945 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:39.960233 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd39045-f3d7-42fc-82cb-94ba8deee945 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:39.961650 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd39045-f3d7-42fc-82cb-94ba8deee945 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:39.971042 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 582/1150] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:04:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:39.978104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b4fe7-df18-4fda-bc54-64640d44fcce None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:39.988328 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:40.088075 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b4fe7-df18-4fda-bc54-64640d44fcce None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:40.128151 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c62d733-0828-4f96-8020-8dfd90f518d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:40.129090 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c62d733-0828-4f96-8020-8dfd90f518d8 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:40.129370 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c62d733-0828-4f96-8020-8dfd90f518d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:40.129643 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c62d733-0828-4f96-8020-8dfd90f518d8 None admin] PATH_INFO: `/v3/users/794f41c47bb34760af2f898710712869` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:40.131004 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c62d733-0828-4f96-8020-8dfd90f518d8 None admin] RBAC: Authorizing `identity:get_user(user_id=794f41c47bb34760af2f898710712869)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:40.132493 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c62d733-0828-4f96-8020-8dfd90f518d8 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:40.134702 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 569/1151] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:04:39 2022] GET /identity/v3/users/794f41c47bb34760af2f898710712869 => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:40.140825 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd39045-f3d7-42fc-82cb-94ba8deee945 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:40.180619 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:40.254587 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd39045-f3d7-42fc-82cb-94ba8deee945 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:40.290317 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b75152-8eb2-4b00-b7a0-ee74a33cd944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:40.291394 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b75152-8eb2-4b00-b7a0-ee74a33cd944 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:40.291677 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b75152-8eb2-4b00-b7a0-ee74a33cd944 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:40.291949 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b75152-8eb2-4b00-b7a0-ee74a33cd944 None admin] PATH_INFO: `/v3/projects/d2a30d1217f447ca911d699505efc965` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:40.293196 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b75152-8eb2-4b00-b7a0-ee74a33cd944 None admin] RBAC: Authorizing `identity:get_project(project_id=d2a30d1217f447ca911d699505efc965)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:40.294643 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b75152-8eb2-4b00-b7a0-ee74a33cd944 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:40.296364 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 583/1152] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:04:40 2022] GET /identity/v3/projects/d2a30d1217f447ca911d699505efc965 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:40.302686 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c62d733-0828-4f96-8020-8dfd90f518d8 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:40.382072 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:40.409063 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c62d733-0828-4f96-8020-8dfd90f518d8 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:40.444711 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb53286-b699-491d-9ef3-26c657252817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:40.445630 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb53286-b699-491d-9ef3-26c657252817 None admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:40.445916 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb53286-b699-491d-9ef3-26c657252817 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:40.446191 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb53286-b699-491d-9ef3-26c657252817 None admin] PATH_INFO: `/v3/projects/d2a30d1217f447ca911d699505efc965/users/794f41c47bb34760af2f898710712869/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:40.448258 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb53286-b699-491d-9ef3-26c657252817 None admin] RBAC: Authorizing `identity:create_grant(project_id=d2a30d1217f447ca911d699505efc965, user_id=794f41c47bb34760af2f898710712869, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:40.449765 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb53286-b699-491d-9ef3-26c657252817 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:40.464986 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 570/1153] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 27 21:04:40 2022] PUT /identity/v3/projects/d2a30d1217f447ca911d699505efc965/users/794f41c47bb34760af2f898710712869/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:04:42.498294 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7f0db-0c0d-443b-ab45-8fc4fd4d21e7 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:42.499158 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7f0db-0c0d-443b-ab45-8fc4fd4d21e7 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:42.499158 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7f0db-0c0d-443b-ab45-8fc4fd4d21e7 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:42.501219 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 584/1154] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:04:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:42.507524 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea8cd0-d081-4946-b0d7-1dd2a63642c1 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:42.508243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea8cd0-d081-4946-b0d7-1dd2a63642c1 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:42.508407 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea8cd0-d081-4946-b0d7-1dd2a63642c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:42.531712 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-35ea8cd0-d081-4946-b0d7-1dd2a63642c1 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:42.532916 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-35ea8cd0-d081-4946-b0d7-1dd2a63642c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:42.607242 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 571/1155] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:42.631217 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7f0db-0c0d-443b-ab45-8fc4fd4d21e7 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:42.749179 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7f0db-0c0d-443b-ab45-8fc4fd4d21e7 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:42.752278 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:42.785623 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94dc093-31dd-4ec2-839c-63250bea5f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:42.786388 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94dc093-31dd-4ec2-839c-63250bea5f60 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:42.786666 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94dc093-31dd-4ec2-839c-63250bea5f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:42.786939 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94dc093-31dd-4ec2-839c-63250bea5f60 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:42.798663 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94dc093-31dd-4ec2-839c-63250bea5f60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:04:42.806455 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-e94dc093-31dd-4ec2-839c-63250bea5f60 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:04:42.806455 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:42.806455 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:42.806455 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:42.806455 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:42.806455 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:42.806455 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:42.806455 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:42.806455 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:42.806455 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:42.806455 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:42.806455 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:42.806455 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:04:42.806455 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:04:42.806455 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:04:42.806455 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:04:42.807575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:04:42.807575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:04:42.807575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:04:42.807575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:04:42.807575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:42.807575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:42.807575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:42.807575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:42.807575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:42.807575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:42.807575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:42.807575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:42.807575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:42.807575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:42.807575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:42.807575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:42.808600 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:42.808600 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:42.808600 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:42.808600 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:42.808600 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:04:42.808600 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:04:42.808600 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:04:42.808600 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:04:42.808600 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:04:42.808600 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:04:42.808600 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:04:42.808600 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:04:42.808600 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 585/1156] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 27 21:04:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:42.814023 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea8cd0-d081-4946-b0d7-1dd2a63642c1 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:42.876742 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:42.917246 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea8cd0-d081-4946-b0d7-1dd2a63642c1 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:42.953472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf0bb9-f5c0-4c97-a18c-b3951319022e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:42.954146 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf0bb9-f5c0-4c97-a18c-b3951319022e None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:42.954431 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf0bb9-f5c0-4c97-a18c-b3951319022e None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:42.954703 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf0bb9-f5c0-4c97-a18c-b3951319022e None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:42.955461 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf0bb9-f5c0-4c97-a18c-b3951319022e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:42.955770 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf0bb9-f5c0-4c97-a18c-b3951319022e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:42.957211 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf0bb9-f5c0-4c97-a18c-b3951319022e None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:42.965704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 572/1157] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:04:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:42.971854 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94dc093-31dd-4ec2-839c-63250bea5f60 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:43.083558 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94dc093-31dd-4ec2-839c-63250bea5f60 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:43.120309 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c491cdd-a6db-4656-b596-da0246428ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:43.121213 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c491cdd-a6db-4656-b596-da0246428ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:43.121490 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c491cdd-a6db-4656-b596-da0246428ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:43.121760 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c491cdd-a6db-4656-b596-da0246428ea5 None admin] PATH_INFO: `/v3/users/794f41c47bb34760af2f898710712869` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:43.122997 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c491cdd-a6db-4656-b596-da0246428ea5 None admin] RBAC: Authorizing `identity:get_user(user_id=794f41c47bb34760af2f898710712869)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:43.124450 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c491cdd-a6db-4656-b596-da0246428ea5 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:43.126129 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 586/1158] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:04:42 2022] GET /identity/v3/users/794f41c47bb34760af2f898710712869 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:43.132262 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf0bb9-f5c0-4c97-a18c-b3951319022e None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:43.238289 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf0bb9-f5c0-4c97-a18c-b3951319022e None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:43.264618 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:43.274992 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ebe4d-dd3b-4126-a691-a08791bea3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:43.275680 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ebe4d-dd3b-4126-a691-a08791bea3ff None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:43.275967 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ebe4d-dd3b-4126-a691-a08791bea3ff None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:43.276243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ebe4d-dd3b-4126-a691-a08791bea3ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:43.351421 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ebe4d-dd3b-4126-a691-a08791bea3ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:43.352966 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ebe4d-dd3b-4126-a691-a08791bea3ff None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:43.461623 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 573/1159] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:04:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 21:04:43.468574 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c491cdd-a6db-4656-b596-da0246428ea5 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:43.505285 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:43.576184 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c491cdd-a6db-4656-b596-da0246428ea5 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:43.614995 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e6da8-8031-4afd-a4bf-bd09db21c266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:43.615872 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e6da8-8031-4afd-a4bf-bd09db21c266 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:43.616155 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e6da8-8031-4afd-a4bf-bd09db21c266 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:43.616430 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e6da8-8031-4afd-a4bf-bd09db21c266 None admin] PATH_INFO: `/v3/projects/d2a30d1217f447ca911d699505efc965` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:43.619400 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4e6da8-8031-4afd-a4bf-bd09db21c266 None admin] RBAC: Authorizing `identity:get_project(project_id=d2a30d1217f447ca911d699505efc965)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:43.620869 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4e6da8-8031-4afd-a4bf-bd09db21c266 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:43.622722 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 587/1160] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:04:43 2022] GET /identity/v3/projects/d2a30d1217f447ca911d699505efc965 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:43.629783 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ebe4d-dd3b-4126-a691-a08791bea3ff None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:43.661050 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:43.745095 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ebe4d-dd3b-4126-a691-a08791bea3ff None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:43.785616 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379fc0ef-8efc-4f94-bf49-babd5e35688b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:43.786271 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379fc0ef-8efc-4f94-bf49-babd5e35688b None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:43.786548 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379fc0ef-8efc-4f94-bf49-babd5e35688b None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:43.786820 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379fc0ef-8efc-4f94-bf49-babd5e35688b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:43.787652 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379fc0ef-8efc-4f94-bf49-babd5e35688b None admin] RBAC: Adding query filter params (role.id=d336dd7b746c4488b33289c00302e4d5, scope.project.id=d2a30d1217f447ca911d699505efc965, user.id=794f41c47bb34760af2f898710712869) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:43.787959 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379fc0ef-8efc-4f94-bf49-babd5e35688b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:43.789432 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379fc0ef-8efc-4f94-bf49-babd5e35688b None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:43.796306 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 574/1161] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:04:43 2022] GET /identity/v3/role_assignments?user.id=794f41c47bb34760af2f898710712869&scope.project.id=d2a30d1217f447ca911d699505efc965&role.id=d336dd7b746c4488b33289c00302e4d5 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:45.841518 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f273d2-c54d-4c4b-a0c3-ad471b65505c None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:45.842145 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f273d2-c54d-4c4b-a0c3-ad471b65505c None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:45.842145 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f273d2-c54d-4c4b-a0c3-ad471b65505c None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:45.843432 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 588/1162] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:45.848849 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f51b09-56b9-47c7-ac20-71835e4e617e None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:45.849347 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f51b09-56b9-47c7-ac20-71835e4e617e None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:45.849448 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f51b09-56b9-47c7-ac20-71835e4e617e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:45.869815 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-53f51b09-56b9-47c7-ac20-71835e4e617e None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:45.871018 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-53f51b09-56b9-47c7-ac20-71835e4e617e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:45.943070 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 575/1163] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:45.968124 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f273d2-c54d-4c4b-a0c3-ad471b65505c None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:46.088239 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:46.092752 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f273d2-c54d-4c4b-a0c3-ad471b65505c None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:46.128799 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f02e6-f940-4a67-95a7-4b856be3baed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:46.129480 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f02e6-f940-4a67-95a7-4b856be3baed None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:46.129760 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f02e6-f940-4a67-95a7-4b856be3baed None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:46.130033 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f02e6-f940-4a67-95a7-4b856be3baed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:46.201364 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28f02e6-f940-4a67-95a7-4b856be3baed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:46.202845 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28f02e6-f940-4a67-95a7-4b856be3baed None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:46.325075 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:46.329777 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 589/1164] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:04:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 21:04:46.336481 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f51b09-56b9-47c7-ac20-71835e4e617e None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:46.362405 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:46.455414 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f51b09-56b9-47c7-ac20-71835e4e617e None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:46.491922 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c367953-2998-4350-a4fa-26658f5583b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:46.492726 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c367953-2998-4350-a4fa-26658f5583b8 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:46.493034 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c367953-2998-4350-a4fa-26658f5583b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:46.493344 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c367953-2998-4350-a4fa-26658f5583b8 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:46.501302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c367953-2998-4350-a4fa-26658f5583b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:46.502757 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c367953-2998-4350-a4fa-26658f5583b8 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:46.517905 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-1c367953-2998-4350-a4fa-26658f5583b8 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 27 21:04:46.517905 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:46.517905 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 27 21:04:46.517905 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 27 21:04:46.517905 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:04:46.517905 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:04:46.517905 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:04:46.517905 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:04:46.517905 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 27 21:04:46.517905 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:04:46.517905 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:04:46.517905 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:04:46.517905 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:46.517905 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:46.517905 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:46.517905 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:46.517905 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:46.519148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:46.519148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:46.519148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:46.519148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:46.519148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:46.519148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:46.519148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:04:46.519148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:04:46.519148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:04:46.519148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:04:46.519148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:46.519148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:46.519148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:46.519148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:46.519148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:46.519148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:46.520273 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:46.520273 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:46.520273 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:46.520273 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:46.520273 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:46.520273 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:46.520273 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:46.520273 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:46.520273 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:46.520273 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:46.520273 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 27 21:04:46.520273 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:04:46.520273 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 27 21:04:46.520273 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:04:46.520273 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 576/1165] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Jun 27 21:04:46 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:46.525537 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f02e6-f940-4a67-95a7-4b856be3baed None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:46.629489 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f02e6-f940-4a67-95a7-4b856be3baed None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:46.665892 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891ae6ce-55de-4437-bca6-0ea9e8566809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:46.666559 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891ae6ce-55de-4437-bca6-0ea9e8566809 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:46.666841 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891ae6ce-55de-4437-bca6-0ea9e8566809 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:46.667113 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891ae6ce-55de-4437-bca6-0ea9e8566809 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:46.667840 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891ae6ce-55de-4437-bca6-0ea9e8566809 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:46.668146 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891ae6ce-55de-4437-bca6-0ea9e8566809 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:46.669561 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891ae6ce-55de-4437-bca6-0ea9e8566809 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:46.677004 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 590/1166] 10.222.0.13 () {56 vars in 1115 bytes} [Mon Jun 27 21:04:46 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:48.704057 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17370543-4641-4d69-bfed-620d7a2712c8 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:48.704634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17370543-4641-4d69-bfed-620d7a2712c8 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:48.704634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17370543-4641-4d69-bfed-620d7a2712c8 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:48.706615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 577/1167] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:04:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:48.715733 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023b992-b2ae-413d-bb08-0ce02cb676c1 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:48.716248 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023b992-b2ae-413d-bb08-0ce02cb676c1 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:48.716248 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023b992-b2ae-413d-bb08-0ce02cb676c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:48.734621 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-1023b992-b2ae-413d-bb08-0ce02cb676c1 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:48.735817 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-1023b992-b2ae-413d-bb08-0ce02cb676c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:48.799856 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:48.808733 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 591/1168] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:48.833970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17370543-4641-4d69-bfed-620d7a2712c8 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:48.934962 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:48.956595 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17370543-4641-4d69-bfed-620d7a2712c8 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:48.994854 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c948d-64d7-4108-8b83-ae62a5178b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:48.995520 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c948d-64d7-4108-8b83-ae62a5178b51 None admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:48.995775 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c948d-64d7-4108-8b83-ae62a5178b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:48.996077 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c948d-64d7-4108-8b83-ae62a5178b51 None admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:48.996841 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47c948d-64d7-4108-8b83-ae62a5178b51 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:48.998259 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47c948d-64d7-4108-8b83-ae62a5178b51 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:49.017162 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 578/1169] 10.222.0.13 () {60 vars in 1138 bytes} [Mon Jun 27 21:04:48 2022] POST /identity/v3/domains => generated 267 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:04:51.063856 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743bae3-3db2-4450-bf48-8f54c00acb90 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:51.064420 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743bae3-3db2-4450-bf48-8f54c00acb90 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:51.064420 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743bae3-3db2-4450-bf48-8f54c00acb90 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:51.065874 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 592/1170] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:04:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:51.071450 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca1a5f-8256-4592-8a01-7dc1b62bcc1e None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:51.071966 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca1a5f-8256-4592-8a01-7dc1b62bcc1e None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:51.072042 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ca1a5f-8256-4592-8a01-7dc1b62bcc1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:51.090924 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-49ca1a5f-8256-4592-8a01-7dc1b62bcc1e None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:51.092139 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-49ca1a5f-8256-4592-8a01-7dc1b62bcc1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:51.162602 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 579/1171] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:51.186933 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e743bae3-3db2-4450-bf48-8f54c00acb90 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:51.291931 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e743bae3-3db2-4450-bf48-8f54c00acb90 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:51.326745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f49515e-d4b7-4a45-ac03-e9fa555588bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:51.327610 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49515e-d4b7-4a45-ac03-e9fa555588bb None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:51.327887 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49515e-d4b7-4a45-ac03-e9fa555588bb None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:51.328141 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49515e-d4b7-4a45-ac03-e9fa555588bb None admin] PATH_INFO: `/v3/domains/de47cd3292244800947761be113790d8` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:51.338888 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f49515e-d4b7-4a45-ac03-e9fa555588bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=de47cd3292244800947761be113790d8)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:51.340344 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f49515e-d4b7-4a45-ac03-e9fa555588bb None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:51.342006 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 593/1172] 10.222.0.13 () {56 vars in 1183 bytes} [Mon Jun 27 21:04:51 2022] GET /identity/v3/domains/de47cd3292244800947761be113790d8 => generated 267 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:51.348217 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ca1a5f-8256-4592-8a01-7dc1b62bcc1e None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:51.417426 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:51.450149 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ca1a5f-8256-4592-8a01-7dc1b62bcc1e None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:51.487426 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35cba08-b370-49b8-a524-866891bf7397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:51.488122 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35cba08-b370-49b8-a524-866891bf7397 None admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:51.488406 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35cba08-b370-49b8-a524-866891bf7397 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:51.488680 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35cba08-b370-49b8-a524-866891bf7397 None admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:51.489409 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35cba08-b370-49b8-a524-866891bf7397 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:51.490864 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35cba08-b370-49b8-a524-866891bf7397 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:51.516818 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 580/1173] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Jun 27 21:04:51 2022] POST /identity/v3/projects => generated 361 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:04:53.572399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfbb224-42c1-4129-b41c-7ecfcb7b92d0 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:53.573030 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfbb224-42c1-4129-b41c-7ecfcb7b92d0 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:53.573030 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfbb224-42c1-4129-b41c-7ecfcb7b92d0 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:53.574434 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 594/1174] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:04:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:53.580232 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ead32-c509-427e-baf1-ebd352b69f57 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:53.580712 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ead32-c509-427e-baf1-ebd352b69f57 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:53.580822 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ead32-c509-427e-baf1-ebd352b69f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:53.598370 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-793ead32-c509-427e-baf1-ebd352b69f57 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:53.599594 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-793ead32-c509-427e-baf1-ebd352b69f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:53.681185 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 581/1175] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:53.706216 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfbb224-42c1-4129-b41c-7ecfcb7b92d0 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:53.766834 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:53.815762 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfbb224-42c1-4129-b41c-7ecfcb7b92d0 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:53.850942 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a577cc9b-a618-4875-9675-1849b2767dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:53.851725 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a577cc9b-a618-4875-9675-1849b2767dfa None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:53.852012 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a577cc9b-a618-4875-9675-1849b2767dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:53.852282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a577cc9b-a618-4875-9675-1849b2767dfa None admin] PATH_INFO: `/v3/domains/de47cd3292244800947761be113790d8` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:53.853613 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a577cc9b-a618-4875-9675-1849b2767dfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=de47cd3292244800947761be113790d8)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:53.855061 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a577cc9b-a618-4875-9675-1849b2767dfa None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:53.856688 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 595/1176] 10.222.0.13 () {56 vars in 1183 bytes} [Mon Jun 27 21:04:53 2022] GET /identity/v3/domains/de47cd3292244800947761be113790d8 => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:53.862807 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793ead32-c509-427e-baf1-ebd352b69f57 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:53.915163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:53.966546 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793ead32-c509-427e-baf1-ebd352b69f57 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:54.001741 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a8f4b-3dcd-40b6-b4ab-a515ceb2c7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:54.002397 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a8f4b-3dcd-40b6-b4ab-a515ceb2c7e1 None admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:54.002677 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a8f4b-3dcd-40b6-b4ab-a515ceb2c7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:54.002946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a8f4b-3dcd-40b6-b4ab-a515ceb2c7e1 None admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:54.003641 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4a8f4b-3dcd-40b6-b4ab-a515ceb2c7e1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:54.005096 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4a8f4b-3dcd-40b6-b4ab-a515ceb2c7e1 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:54.027047 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 582/1177] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Jun 27 21:04:53 2022] POST /identity/v3/users => generated 311 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:04:56.112482 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91ce9e3-dfe1-45d6-8ba3-5c1114b4a33e None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:56.113066 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91ce9e3-dfe1-45d6-8ba3-5c1114b4a33e None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:56.113066 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91ce9e3-dfe1-45d6-8ba3-5c1114b4a33e None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:56.114443 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 596/1178] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:56.119884 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f9292-8aa1-440e-819a-6fcaa599a0bf None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:56.120331 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f9292-8aa1-440e-819a-6fcaa599a0bf None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:56.120422 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f9292-8aa1-440e-819a-6fcaa599a0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:56.138253 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-b42f9292-8aa1-440e-819a-6fcaa599a0bf None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:56.139454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-b42f9292-8aa1-440e-819a-6fcaa599a0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:56.208446 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 583/1179] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:56.233347 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91ce9e3-dfe1-45d6-8ba3-5c1114b4a33e None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:56.338454 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91ce9e3-dfe1-45d6-8ba3-5c1114b4a33e None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:56.361616 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:56.373969 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be4727-25cc-4e68-8a79-9b14aaff3b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:56.374750 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be4727-25cc-4e68-8a79-9b14aaff3b13 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:56.375025 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be4727-25cc-4e68-8a79-9b14aaff3b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:56.375301 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be4727-25cc-4e68-8a79-9b14aaff3b13 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:56.386232 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3be4727-25cc-4e68-8a79-9b14aaff3b13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:04:56.393827 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-f3be4727-25cc-4e68-8a79-9b14aaff3b13 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:04:56.393827 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:56.393827 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:56.393827 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:56.393827 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:56.393827 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:56.393827 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:56.393827 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:56.393827 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:56.393827 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:56.393827 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:56.393827 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:56.393827 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:04:56.393827 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:04:56.393827 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:04:56.393827 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:04:56.394992 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:04:56.394992 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:04:56.394992 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:04:56.394992 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:04:56.394992 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:56.394992 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:56.394992 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:56.394992 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:56.394992 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:56.394992 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:56.394992 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:56.394992 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:56.394992 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:56.394992 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:56.394992 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:56.394992 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:56.395918 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:56.395918 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:56.395918 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:56.395918 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:56.395918 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:04:56.395918 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:04:56.395918 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:04:56.395918 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:04:56.395918 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:04:56.395918 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:04:56.395918 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:04:56.395918 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:04:56.395918 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 597/1180] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 27 21:04:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:56.401506 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f9292-8aa1-440e-819a-6fcaa599a0bf None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:56.443599 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:56.507108 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f9292-8aa1-440e-819a-6fcaa599a0bf None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:56.542497 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14180eb-132b-4bc7-838e-dbdd046c0d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:56.543158 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14180eb-132b-4bc7-838e-dbdd046c0d10 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:56.543443 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14180eb-132b-4bc7-838e-dbdd046c0d10 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:56.543719 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14180eb-132b-4bc7-838e-dbdd046c0d10 None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:56.544472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14180eb-132b-4bc7-838e-dbdd046c0d10 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:56.544783 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14180eb-132b-4bc7-838e-dbdd046c0d10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:56.546190 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14180eb-132b-4bc7-838e-dbdd046c0d10 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:56.554438 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 584/1181] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:04:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:56.560638 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be4727-25cc-4e68-8a79-9b14aaff3b13 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:56.666970 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be4727-25cc-4e68-8a79-9b14aaff3b13 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:56.704569 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e660540-ce3c-4bfb-91ab-abdf8c042289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:56.705469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e660540-ce3c-4bfb-91ab-abdf8c042289 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:56.705748 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e660540-ce3c-4bfb-91ab-abdf8c042289 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:56.706015 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e660540-ce3c-4bfb-91ab-abdf8c042289 None admin] PATH_INFO: `/v3/users/88fb343c9ece40e9a47a087d3f68e3bb` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:56.721411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e660540-ce3c-4bfb-91ab-abdf8c042289 None admin] RBAC: Authorizing `identity:get_user(user_id=88fb343c9ece40e9a47a087d3f68e3bb)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:56.722871 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e660540-ce3c-4bfb-91ab-abdf8c042289 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:56.724626 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 598/1182] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:04:56 2022] GET /identity/v3/users/88fb343c9ece40e9a47a087d3f68e3bb => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:56.731542 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14180eb-132b-4bc7-838e-dbdd046c0d10 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:56.833409 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:56.836448 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14180eb-132b-4bc7-838e-dbdd046c0d10 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:56.873172 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5df1d4-ff71-4cab-8f42-21591b145145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:56.873838 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5df1d4-ff71-4cab-8f42-21591b145145 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:56.874116 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5df1d4-ff71-4cab-8f42-21591b145145 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:56.874385 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5df1d4-ff71-4cab-8f42-21591b145145 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:56.944977 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5df1d4-ff71-4cab-8f42-21591b145145 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:56.946344 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5df1d4-ff71-4cab-8f42-21591b145145 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:57.045628 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:57.049655 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 585/1183] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:04:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 21:04:57.055640 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e660540-ce3c-4bfb-91ab-abdf8c042289 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:57.100970 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:57.162215 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e660540-ce3c-4bfb-91ab-abdf8c042289 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:57.197598 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a718485-2b2d-435a-8637-524c3ad8cfe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:57.198378 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a718485-2b2d-435a-8637-524c3ad8cfe7 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:57.198655 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a718485-2b2d-435a-8637-524c3ad8cfe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:57.198922 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a718485-2b2d-435a-8637-524c3ad8cfe7 None admin] PATH_INFO: `/v3/projects/315867fbaf354ebd97c7419397257d5c` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:57.200134 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a718485-2b2d-435a-8637-524c3ad8cfe7 None admin] RBAC: Authorizing `identity:get_project(project_id=315867fbaf354ebd97c7419397257d5c)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:57.201618 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a718485-2b2d-435a-8637-524c3ad8cfe7 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:57.203317 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 599/1184] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:04:57 2022] GET /identity/v3/projects/315867fbaf354ebd97c7419397257d5c => generated 361 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:57.209783 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5df1d4-ff71-4cab-8f42-21591b145145 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:57.311601 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5df1d4-ff71-4cab-8f42-21591b145145 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:57.346536 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcead6f-e3a9-4c86-b95a-c975707e39ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:57.347178 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcead6f-e3a9-4c86-b95a-c975707e39ec None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:57.347457 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcead6f-e3a9-4c86-b95a-c975707e39ec None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:57.347724 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcead6f-e3a9-4c86-b95a-c975707e39ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:57.348544 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcead6f-e3a9-4c86-b95a-c975707e39ec None admin] RBAC: Adding query filter params (role.id=d336dd7b746c4488b33289c00302e4d5, scope.project.id=315867fbaf354ebd97c7419397257d5c, user.id=88fb343c9ece40e9a47a087d3f68e3bb) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:57.348845 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcead6f-e3a9-4c86-b95a-c975707e39ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:57.350644 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcead6f-e3a9-4c86-b95a-c975707e39ec None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:57.356255 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 586/1185] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:04:57 2022] GET /identity/v3/role_assignments?user.id=88fb343c9ece40e9a47a087d3f68e3bb&scope.project.id=315867fbaf354ebd97c7419397257d5c&role.id=d336dd7b746c4488b33289c00302e4d5 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:59.336079 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad85227-a7d9-4737-be98-724f30e6607f None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:59.336612 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad85227-a7d9-4737-be98-724f30e6607f None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:59.336612 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad85227-a7d9-4737-be98-724f30e6607f None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:59.338057 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 600/1186] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:04:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:04:59.343691 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc8a86-5c71-4a22-9632-309a4d5e5ef4 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:59.344151 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc8a86-5c71-4a22-9632-309a4d5e5ef4 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:59.344261 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc8a86-5c71-4a22-9632-309a4d5e5ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:59.363116 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-58dc8a86-5c71-4a22-9632-309a4d5e5ef4 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:04:59.364301 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-58dc8a86-5c71-4a22-9632-309a4d5e5ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:04:59.430285 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:59.439878 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 587/1187] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:04:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:04:59.463583 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad85227-a7d9-4737-be98-724f30e6607f None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:59.572423 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad85227-a7d9-4737-be98-724f30e6607f None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:59.582014 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:04:59.609567 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578a206-3858-4f58-b30c-32ef05f26546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:59.610276 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578a206-3858-4f58-b30c-32ef05f26546 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:59.610552 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578a206-3858-4f58-b30c-32ef05f26546 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:59.610822 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578a206-3858-4f58-b30c-32ef05f26546 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:59.622184 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a578a206-3858-4f58-b30c-32ef05f26546 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:04:59.629709 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-a578a206-3858-4f58-b30c-32ef05f26546 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:04:59.629709 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:04:59.629709 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:04:59.629709 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:04:59.629709 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:04:59.629709 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:04:59.629709 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:04:59.629709 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:04:59.629709 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:04:59.629709 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:04:59.629709 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:04:59.629709 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:04:59.629709 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:04:59.629709 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:04:59.629709 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:04:59.629709 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:04:59.630674 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:04:59.630674 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:04:59.630674 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:04:59.630674 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:04:59.630674 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:04:59.630674 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:04:59.630674 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:04:59.630674 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:04:59.630674 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:04:59.630674 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:04:59.630674 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:04:59.630674 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:04:59.630674 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:04:59.630674 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:04:59.630674 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:04:59.630674 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:04:59.631554 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:04:59.631554 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:04:59.631554 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:04:59.631554 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:04:59.631554 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:04:59.631554 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:04:59.631554 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:04:59.631554 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:04:59.631554 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:04:59.631554 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:04:59.631554 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:04:59.631554 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:04:59.631554 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 601/1188] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 27 21:04:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:04:59.637760 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc8a86-5c71-4a22-9632-309a4d5e5ef4 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:59.744057 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc8a86-5c71-4a22-9632-309a4d5e5ef4 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:59.780065 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a5706c-dfbf-409e-901a-3843c4a57ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:59.780733 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5706c-dfbf-409e-901a-3843c4a57ed9 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:59.781039 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5706c-dfbf-409e-901a-3843c4a57ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:59.781314 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5706c-dfbf-409e-901a-3843c4a57ed9 None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:59.782067 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a5706c-dfbf-409e-901a-3843c4a57ed9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:04:59.782370 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a5706c-dfbf-409e-901a-3843c4a57ed9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:59.783745 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a5706c-dfbf-409e-901a-3843c4a57ed9 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:59.793278 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 588/1189] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:04:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:59.799530 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578a206-3858-4f58-b30c-32ef05f26546 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:04:59.905462 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578a206-3858-4f58-b30c-32ef05f26546 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:04:59.942964 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1cb6d-314b-48d0-bc76-0f17da8166bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:04:59.943847 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1cb6d-314b-48d0-bc76-0f17da8166bf None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:04:59.944122 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1cb6d-314b-48d0-bc76-0f17da8166bf None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:04:59.944390 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1cb6d-314b-48d0-bc76-0f17da8166bf None admin] PATH_INFO: `/v3/users/88fb343c9ece40e9a47a087d3f68e3bb` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:04:59.945703 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef1cb6d-314b-48d0-bc76-0f17da8166bf None admin] RBAC: Authorizing `identity:get_user(user_id=88fb343c9ece40e9a47a087d3f68e3bb)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:04:59.947909 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef1cb6d-314b-48d0-bc76-0f17da8166bf None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:04:59.949606 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 602/1190] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:04:59 2022] GET /identity/v3/users/88fb343c9ece40e9a47a087d3f68e3bb => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:04:59.955911 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a5706c-dfbf-409e-901a-3843c4a57ed9 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:00.018316 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:00.067958 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a5706c-dfbf-409e-901a-3843c4a57ed9 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:00.104985 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2346cc-cb8d-42ab-a965-919d41d4f4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:00.105874 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2346cc-cb8d-42ab-a965-919d41d4f4d3 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:00.106152 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2346cc-cb8d-42ab-a965-919d41d4f4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:00.106420 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2346cc-cb8d-42ab-a965-919d41d4f4d3 None admin] PATH_INFO: `/v3/projects/315867fbaf354ebd97c7419397257d5c` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:00.107756 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2346cc-cb8d-42ab-a965-919d41d4f4d3 None admin] RBAC: Authorizing `identity:get_project(project_id=315867fbaf354ebd97c7419397257d5c)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:00.109249 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2346cc-cb8d-42ab-a965-919d41d4f4d3 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:00.111069 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 589/1191] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:04:59 2022] GET /identity/v3/projects/315867fbaf354ebd97c7419397257d5c => generated 361 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:05:00.117362 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1cb6d-314b-48d0-bc76-0f17da8166bf None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:00.156323 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:00.224136 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1cb6d-314b-48d0-bc76-0f17da8166bf None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:00.260458 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf05b333-3963-4a5e-8de1-1334f6509e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:00.261377 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05b333-3963-4a5e-8de1-1334f6509e26 None admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:00.261665 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05b333-3963-4a5e-8de1-1334f6509e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:00.261939 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf05b333-3963-4a5e-8de1-1334f6509e26 None admin] PATH_INFO: `/v3/projects/315867fbaf354ebd97c7419397257d5c/users/88fb343c9ece40e9a47a087d3f68e3bb/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:00.264157 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf05b333-3963-4a5e-8de1-1334f6509e26 None admin] RBAC: Authorizing `identity:create_grant(project_id=315867fbaf354ebd97c7419397257d5c, user_id=88fb343c9ece40e9a47a087d3f68e3bb, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:00.265654 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf05b333-3963-4a5e-8de1-1334f6509e26 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:00.281784 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 603/1192] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 27 21:05:00 2022] PUT /identity/v3/projects/315867fbaf354ebd97c7419397257d5c/users/88fb343c9ece40e9a47a087d3f68e3bb/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:05:02.319108 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a5953-b2d6-49eb-907e-7f274e48f5af None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:02.319659 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a5953-b2d6-49eb-907e-7f274e48f5af None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:02.319659 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a5953-b2d6-49eb-907e-7f274e48f5af None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:02.320945 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 590/1193] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:05:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:05:02.326499 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a156d-4546-4a8c-b5ec-74fe29e89e05 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:02.326957 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a156d-4546-4a8c-b5ec-74fe29e89e05 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:02.327149 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a156d-4546-4a8c-b5ec-74fe29e89e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:02.345464 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-a91a156d-4546-4a8c-b5ec-74fe29e89e05 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:05:02.346709 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a91a156d-4546-4a8c-b5ec-74fe29e89e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:05:02.419349 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 604/1194] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:05:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:05:02.443320 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a5953-b2d6-49eb-907e-7f274e48f5af None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:02.549517 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a5953-b2d6-49eb-907e-7f274e48f5af None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:02.573932 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:02.585607 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343b0921-dbd4-435a-a4ed-0570ed5acd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:02.586411 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b0921-dbd4-435a-a4ed-0570ed5acd8d None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:02.586694 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b0921-dbd4-435a-a4ed-0570ed5acd8d None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:02.587131 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b0921-dbd4-435a-a4ed-0570ed5acd8d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:02.598672 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343b0921-dbd4-435a-a4ed-0570ed5acd8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:05:02.606672 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-343b0921-dbd4-435a-a4ed-0570ed5acd8d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:05:02.606672 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:05:02.606672 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:05:02.606672 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:05:02.606672 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:05:02.606672 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:05:02.606672 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:05:02.606672 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:05:02.606672 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:05:02.606672 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:05:02.606672 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:05:02.606672 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:05:02.606672 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:05:02.606672 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:05:02.606672 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:05:02.606672 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:05:02.607746 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:05:02.607746 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:05:02.607746 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:05:02.607746 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:05:02.607746 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:05:02.607746 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:05:02.607746 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:05:02.607746 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:05:02.607746 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:05:02.607746 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:05:02.607746 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:05:02.607746 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:05:02.607746 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:05:02.607746 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:05:02.607746 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:05:02.607746 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:05:02.608760 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:05:02.608760 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:05:02.608760 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:05:02.608760 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:05:02.608760 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:05:02.608760 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:05:02.608760 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:05:02.608760 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:05:02.608760 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:05:02.608760 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:05:02.608760 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:05:02.608760 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:05:02.613120 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 591/1195] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 27 21:05:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:05:02.614172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a156d-4546-4a8c-b5ec-74fe29e89e05 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:02.641959 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:02.718406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a156d-4546-4a8c-b5ec-74fe29e89e05 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:02.756005 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3423f5f-11e7-4da8-b0e9-1e8605e3610f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:02.756679 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3423f5f-11e7-4da8-b0e9-1e8605e3610f None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:02.756965 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3423f5f-11e7-4da8-b0e9-1e8605e3610f None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:02.757263 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3423f5f-11e7-4da8-b0e9-1e8605e3610f None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:02.758028 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3423f5f-11e7-4da8-b0e9-1e8605e3610f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:05:02.758339 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3423f5f-11e7-4da8-b0e9-1e8605e3610f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:02.759956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3423f5f-11e7-4da8-b0e9-1e8605e3610f None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:02.768552 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 605/1196] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:05:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:05:02.775007 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343b0921-dbd4-435a-a4ed-0570ed5acd8d None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:02.880131 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343b0921-dbd4-435a-a4ed-0570ed5acd8d None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:02.917903 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3233e9e-5f7a-4a71-a2e3-ee4f9df2ea09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:02.918777 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3233e9e-5f7a-4a71-a2e3-ee4f9df2ea09 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:02.919052 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3233e9e-5f7a-4a71-a2e3-ee4f9df2ea09 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:02.919322 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3233e9e-5f7a-4a71-a2e3-ee4f9df2ea09 None admin] PATH_INFO: `/v3/users/88fb343c9ece40e9a47a087d3f68e3bb` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:02.920545 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3233e9e-5f7a-4a71-a2e3-ee4f9df2ea09 None admin] RBAC: Authorizing `identity:get_user(user_id=88fb343c9ece40e9a47a087d3f68e3bb)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:02.922025 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3233e9e-5f7a-4a71-a2e3-ee4f9df2ea09 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:02.923649 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 592/1197] 10.222.0.13 () {56 vars in 1177 bytes} [Mon Jun 27 21:05:02 2022] GET /identity/v3/users/88fb343c9ece40e9a47a087d3f68e3bb => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:05:02.929701 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3423f5f-11e7-4da8-b0e9-1e8605e3610f None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:03.027298 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:03.044636 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3423f5f-11e7-4da8-b0e9-1e8605e3610f None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:03.079439 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e2034-feff-4b7b-931e-41952567ec74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:03.080089 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e2034-feff-4b7b-931e-41952567ec74 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:03.080373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e2034-feff-4b7b-931e-41952567ec74 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:03.080645 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e2034-feff-4b7b-931e-41952567ec74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:03.154378 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5e2034-feff-4b7b-931e-41952567ec74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:03.155754 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5e2034-feff-4b7b-931e-41952567ec74 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:03.243096 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:03.256669 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 606/1198] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:05:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 21:05:03.263093 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3233e9e-5f7a-4a71-a2e3-ee4f9df2ea09 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:03.319460 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:03.371066 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3233e9e-5f7a-4a71-a2e3-ee4f9df2ea09 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:03.411135 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ffa1b8-65f1-4254-b33a-1b641df06a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:03.412003 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ffa1b8-65f1-4254-b33a-1b641df06a21 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:03.412285 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ffa1b8-65f1-4254-b33a-1b641df06a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:03.412561 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ffa1b8-65f1-4254-b33a-1b641df06a21 None admin] PATH_INFO: `/v3/projects/315867fbaf354ebd97c7419397257d5c` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:03.413824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ffa1b8-65f1-4254-b33a-1b641df06a21 None admin] RBAC: Authorizing `identity:get_project(project_id=315867fbaf354ebd97c7419397257d5c)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:03.415369 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ffa1b8-65f1-4254-b33a-1b641df06a21 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:03.417114 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 593/1199] 10.222.0.13 () {56 vars in 1186 bytes} [Mon Jun 27 21:05:03 2022] GET /identity/v3/projects/315867fbaf354ebd97c7419397257d5c => generated 361 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:05:03.423562 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e2034-feff-4b7b-931e-41952567ec74 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:03.527027 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e2034-feff-4b7b-931e-41952567ec74 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:03.562340 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbaead4-eb52-407f-a061-e13cb4752b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:03.562989 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbaead4-eb52-407f-a061-e13cb4752b25 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:03.563267 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbaead4-eb52-407f-a061-e13cb4752b25 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:03.563547 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbaead4-eb52-407f-a061-e13cb4752b25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:03.564381 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbaead4-eb52-407f-a061-e13cb4752b25 None admin] RBAC: Adding query filter params (role.id=d336dd7b746c4488b33289c00302e4d5, scope.project.id=315867fbaf354ebd97c7419397257d5c, user.id=88fb343c9ece40e9a47a087d3f68e3bb) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:05:03.564693 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbaead4-eb52-407f-a061-e13cb4752b25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:03.566177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbaead4-eb52-407f-a061-e13cb4752b25 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:05:03.572277 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 607/1200] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:05:03 2022] GET /identity/v3/role_assignments?user.id=88fb343c9ece40e9a47a087d3f68e3bb&scope.project.id=315867fbaf354ebd97c7419397257d5c&role.id=d336dd7b746c4488b33289c00302e4d5 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:05:13.406501 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=40 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:13.569424 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=74 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:05:59.628184 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21badb6-fd25-4211-baf7-f91fa980fbbf None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:59.628184 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21badb6-fd25-4211-baf7-f91fa980fbbf None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:59.628184 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21badb6-fd25-4211-baf7-f91fa980fbbf None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:59.628184 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 594/1201] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:05:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:05:59.635270 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254dca8-7072-4078-81f7-2321d29dd3dd None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:59.635850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254dca8-7072-4078-81f7-2321d29dd3dd None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:59.635850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254dca8-7072-4078-81f7-2321d29dd3dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:59.655440 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-5254dca8-7072-4078-81f7-2321d29dd3dd None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:05:59.656831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-5254dca8-7072-4078-81f7-2321d29dd3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:05:59.751152 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 608/1202] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:05:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:05:59.776815 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21badb6-fd25-4211-baf7-f91fa980fbbf None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:05:59.928391 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21badb6-fd25-4211-baf7-f91fa980fbbf None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:05:59.980046 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8583dfd-9c14-42cb-acb3-1a0be92d0b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:05:59.980918 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8583dfd-9c14-42cb-acb3-1a0be92d0b6f None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:05:59.981277 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8583dfd-9c14-42cb-acb3-1a0be92d0b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:05:59.981892 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8583dfd-9c14-42cb-acb3-1a0be92d0b6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:05:59.994128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8583dfd-9c14-42cb-acb3-1a0be92d0b6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:05:59.995891 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8583dfd-9c14-42cb-acb3-1a0be92d0b6f None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:00.016803 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-c8583dfd-9c14-42cb-acb3-1a0be92d0b6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:00.016803 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:00.016803 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:00.016803 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:00.016803 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:00.016803 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:00.016803 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:00.016803 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:00.016803 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:00.016803 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:00.016803 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:00.016803 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:00.016803 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:06:00.016803 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:06:00.016803 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:06:00.018206 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:06:00.018206 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:00.018206 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:00.018206 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:00.018206 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:00.018206 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:00.018206 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:00.018206 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:00.018206 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:00.018206 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:00.018206 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:00.018206 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:00.018206 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:00.018206 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:00.018206 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:00.018206 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:00.018206 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:00.019776 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:06:00.019776 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:06:00.019776 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:00.019776 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:06:00.019776 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 595/1203] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:05:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:00.025958 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254dca8-7072-4078-81f7-2321d29dd3dd None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:00.155231 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254dca8-7072-4078-81f7-2321d29dd3dd None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:00.189385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:00.197181 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78ab34b-fd5f-4032-8fbc-3feb751399f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:00.198220 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ab34b-fd5f-4032-8fbc-3feb751399f7 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:00.198836 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ab34b-fd5f-4032-8fbc-3feb751399f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:00.199112 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ab34b-fd5f-4032-8fbc-3feb751399f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:00.199952 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78ab34b-fd5f-4032-8fbc-3feb751399f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:00.200290 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78ab34b-fd5f-4032-8fbc-3feb751399f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:00.203726 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78ab34b-fd5f-4032-8fbc-3feb751399f7 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:00.218422 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 609/1204] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:06:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:00.224891 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8583dfd-9c14-42cb-acb3-1a0be92d0b6f None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:00.294445 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:00.363279 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8583dfd-9c14-42cb-acb3-1a0be92d0b6f None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:00.418725 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab9b67c-ee48-450d-a2c8-4db190bf3cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:00.420667 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9b67c-ee48-450d-a2c8-4db190bf3cc8 None admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:00.421674 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9b67c-ee48-450d-a2c8-4db190bf3cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:00.422587 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9b67c-ee48-450d-a2c8-4db190bf3cc8 None admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:00.424341 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab9b67c-ee48-450d-a2c8-4db190bf3cc8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:00.427629 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab9b67c-ee48-450d-a2c8-4db190bf3cc8 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:00.473063 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 596/1205] 10.222.0.13 () {60 vars in 1132 bytes} [Mon Jun 27 21:06:00 2022] POST /identity/v3/users => generated 251 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:06:02.708318 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b1011-d0c7-4265-a754-695827d61cb5 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:02.708877 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b1011-d0c7-4265-a754-695827d61cb5 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:02.709010 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b1011-d0c7-4265-a754-695827d61cb5 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:02.710522 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 610/1206] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:06:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:06:02.716716 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4cc6eb-a93e-46e3-a1ba-1690e1f70d84 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:02.717248 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4cc6eb-a93e-46e3-a1ba-1690e1f70d84 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:02.717326 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4cc6eb-a93e-46e3-a1ba-1690e1f70d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:02.736465 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-0f4cc6eb-a93e-46e3-a1ba-1690e1f70d84 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:06:02.737811 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-0f4cc6eb-a93e-46e3-a1ba-1690e1f70d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:06:02.832278 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 597/1207] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:06:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:06:02.858861 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b1011-d0c7-4265-a754-695827d61cb5 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:02.977337 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b1011-d0c7-4265-a754-695827d61cb5 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:03.018599 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fd3a9e-8600-4db4-8323-7e638b2f44a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:03.019706 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd3a9e-8600-4db4-8323-7e638b2f44a4 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:03.020203 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd3a9e-8600-4db4-8323-7e638b2f44a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:03.020675 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fd3a9e-8600-4db4-8323-7e638b2f44a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:03.038829 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fd3a9e-8600-4db4-8323-7e638b2f44a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:06:03.047356 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-25fd3a9e-8600-4db4-8323-7e638b2f44a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:06:03.047356 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:03.047356 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:03.047356 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:03.047356 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:03.047356 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:03.047356 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:03.047356 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:03.047356 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:03.047356 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:03.047356 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:03.047356 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:03.047356 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:06:03.047356 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:06:03.047356 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:06:03.047356 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:06:03.048547 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:06:03.048547 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:06:03.048547 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:06:03.048547 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:06:03.048547 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:03.048547 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:03.048547 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:03.048547 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:03.048547 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:03.048547 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:03.048547 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:03.048547 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:03.048547 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:03.048547 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:03.048547 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:03.048547 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:03.049597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:03.049597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:03.049597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:03.049597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:03.049597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:06:03.049597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:06:03.049597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:06:03.049597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:06:03.049597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:06:03.049597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:06:03.049597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:06:03.049597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:06:03.056948 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 611/1208] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 27 21:06:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:03.058223 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4cc6eb-a93e-46e3-a1ba-1690e1f70d84 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:03.107893 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:03.188362 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4cc6eb-a93e-46e3-a1ba-1690e1f70d84 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:03.237506 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c3c24-98f9-4aa8-bdc8-a2cb83d3ffdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:03.238560 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c3c24-98f9-4aa8-bdc8-a2cb83d3ffdc None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:03.238947 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c3c24-98f9-4aa8-bdc8-a2cb83d3ffdc None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:03.239341 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c3c24-98f9-4aa8-bdc8-a2cb83d3ffdc None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:03.240210 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794c3c24-98f9-4aa8-bdc8-a2cb83d3ffdc None admin] RBAC: Adding query filter params (name=service) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:03.240620 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794c3c24-98f9-4aa8-bdc8-a2cb83d3ffdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:03.242728 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794c3c24-98f9-4aa8-bdc8-a2cb83d3ffdc None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:03.253252 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 598/1209] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 27 21:06:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:03.260138 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fd3a9e-8600-4db4-8323-7e638b2f44a4 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:03.315793 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:03.375589 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fd3a9e-8600-4db4-8323-7e638b2f44a4 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:03.419161 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9003a94-b88e-46b7-89e3-5820eb628973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:03.419982 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9003a94-b88e-46b7-89e3-5820eb628973 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:03.420375 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9003a94-b88e-46b7-89e3-5820eb628973 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:03.420541 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9003a94-b88e-46b7-89e3-5820eb628973 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:03.431687 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9003a94-b88e-46b7-89e3-5820eb628973 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:03.433162 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9003a94-b88e-46b7-89e3-5820eb628973 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:03.452389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-a9003a94-b88e-46b7-89e3-5820eb628973 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:03.452389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:03.452389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:03.452389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:03.452389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:03.452389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:03.452389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:03.452389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:03.452389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:03.452389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:03.452389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:03.452389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:03.452389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:06:03.452389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:06:03.452389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:06:03.459106 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:06:03.459106 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:03.459106 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:03.459106 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:03.459106 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:03.459106 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:03.459106 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:03.459106 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:03.459106 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:03.459106 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:03.459106 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:03.459106 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:03.459106 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:03.459106 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:03.459106 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:03.459106 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:03.459106 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:03.460146 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:06:03.460146 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:06:03.460146 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:03.460146 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:06:03.460146 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 612/1210] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:06:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:06:03.462268 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c3c24-98f9-4aa8-bdc8-a2cb83d3ffdc None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:03.587092 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:03.590467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c3c24-98f9-4aa8-bdc8-a2cb83d3ffdc None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:03.630234 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d257c-5fdb-4b3d-b4f0-a8987c24853b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:03.631068 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d257c-5fdb-4b3d-b4f0-a8987c24853b None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:03.631361 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d257c-5fdb-4b3d-b4f0-a8987c24853b None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:03.631642 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d257c-5fdb-4b3d-b4f0-a8987c24853b None admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:03.632480 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822d257c-5fdb-4b3d-b4f0-a8987c24853b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:03.632816 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822d257c-5fdb-4b3d-b4f0-a8987c24853b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:03.634517 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822d257c-5fdb-4b3d-b4f0-a8987c24853b None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:03.656042 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 599/1211] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:06:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:03.657294 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9003a94-b88e-46b7-89e3-5820eb628973 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:03.773055 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9003a94-b88e-46b7-89e3-5820eb628973 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:03.775003 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:03.812600 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a95190e-1548-4a19-b92e-4ba796017a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:03.813600 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95190e-1548-4a19-b92e-4ba796017a25 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:03.813887 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95190e-1548-4a19-b92e-4ba796017a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:03.814144 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95190e-1548-4a19-b92e-4ba796017a25 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:03.823164 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a95190e-1548-4a19-b92e-4ba796017a25 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:03.824659 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a95190e-1548-4a19-b92e-4ba796017a25 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:03.842810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-1a95190e-1548-4a19-b92e-4ba796017a25 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 27 21:06:03.842810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:03.842810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:03.842810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:03.842810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:03.842810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:03.842810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:03.842810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:03.842810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:03.842810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:03.842810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:03.842810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:03.842810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:06:03.842810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:06:03.842810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:06:03.842810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:06:03.844191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:03.844191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:03.844191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:06:03.844191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:06:03.844191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:06:03.844191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:06:03.844191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:03.844191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:03.844191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:03.844191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:03.844191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:03.844191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:03.844191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:03.844191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:03.844191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:03.844191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:03.844191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:03.844191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:03.845210 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:03.845210 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:03.845210 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:06:03.845210 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:06:03.845210 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:06:03.845210 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:06:03.845210 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:06:03.845210 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:06:03.845210 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 27 21:06:03.845210 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:06:03.845210 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 613/1212] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:06:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:06:03.850804 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d257c-5fdb-4b3d-b4f0-a8987c24853b None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:03.967199 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d257c-5fdb-4b3d-b4f0-a8987c24853b None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:04.008779 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1741829c-7c42-44be-82cb-4fbe2093ac36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:04.009557 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741829c-7c42-44be-82cb-4fbe2093ac36 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:04.009835 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741829c-7c42-44be-82cb-4fbe2093ac36 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:04.010107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741829c-7c42-44be-82cb-4fbe2093ac36 None admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:04.011053 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1741829c-7c42-44be-82cb-4fbe2093ac36 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:04.011397 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1741829c-7c42-44be-82cb-4fbe2093ac36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:04.013024 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1741829c-7c42-44be-82cb-4fbe2093ac36 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:04.034618 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 600/1213] 10.222.0.13 () {56 vars in 1143 bytes} [Mon Jun 27 21:06:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:04.041422 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a95190e-1548-4a19-b92e-4ba796017a25 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:04.169984 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a95190e-1548-4a19-b92e-4ba796017a25 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:04.208555 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71104dc-88ab-4104-b1cd-c89ade881545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:04.209249 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71104dc-88ab-4104-b1cd-c89ade881545 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:04.209531 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71104dc-88ab-4104-b1cd-c89ade881545 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:04.209837 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71104dc-88ab-4104-b1cd-c89ade881545 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:04.248427 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:04.294311 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71104dc-88ab-4104-b1cd-c89ade881545 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:04.295830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71104dc-88ab-4104-b1cd-c89ade881545 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:04.408880 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 614/1214] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:06:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 21:06:04.415592 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1741829c-7c42-44be-82cb-4fbe2093ac36 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:04.440801 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:04.543815 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1741829c-7c42-44be-82cb-4fbe2093ac36 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:04.587332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e8d3c0-93e7-4e3d-a926-43a4661577f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:04.588060 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e8d3c0-93e7-4e3d-a926-43a4661577f0 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:04.588372 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e8d3c0-93e7-4e3d-a926-43a4661577f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:04.588697 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e8d3c0-93e7-4e3d-a926-43a4661577f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:04.600771 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e8d3c0-93e7-4e3d-a926-43a4661577f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:04.602435 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e8d3c0-93e7-4e3d-a926-43a4661577f0 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:04.624615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-c9e8d3c0-93e7-4e3d-a926-43a4661577f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:04.624615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:04.624615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:04.624615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:04.624615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:04.624615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:04.624615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:04.624615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:04.624615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:04.624615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:04.624615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:04.624615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:04.624615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:06:04.624615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:06:04.624615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:06:04.625648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:06:04.625648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:04.625648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:04.625648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:04.625648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:04.625648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:04.625648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:04.625648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:04.625648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:04.625648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:04.625648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:04.625648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:04.625648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:04.625648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:04.625648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:04.625648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:04.625648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:04.626547 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:06:04.626547 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:06:04.626547 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:04.626547 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:06:04.626547 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 601/1215] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:06:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:04.632845 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71104dc-88ab-4104-b1cd-c89ade881545 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:04.704757 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:04.757123 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71104dc-88ab-4104-b1cd-c89ade881545 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:04.795323 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54b4b9-40f2-48c5-9d50-69d962604d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:04.796028 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54b4b9-40f2-48c5-9d50-69d962604d1e None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:04.796261 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54b4b9-40f2-48c5-9d50-69d962604d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:04.796530 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54b4b9-40f2-48c5-9d50-69d962604d1e None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:04.797307 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54b4b9-40f2-48c5-9d50-69d962604d1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:04.797601 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54b4b9-40f2-48c5-9d50-69d962604d1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:04.799551 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54b4b9-40f2-48c5-9d50-69d962604d1e None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:04.817791 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 615/1216] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:06:04.819286 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e8d3c0-93e7-4e3d-a926-43a4661577f0 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:04.907036 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:04.946112 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e8d3c0-93e7-4e3d-a926-43a4661577f0 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:04.993353 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03dc77-5767-418e-bd49-0a0b6377f35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:04.994229 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03dc77-5767-418e-bd49-0a0b6377f35b None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:04.994589 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03dc77-5767-418e-bd49-0a0b6377f35b None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:04.994871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03dc77-5767-418e-bd49-0a0b6377f35b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:05.002708 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa03dc77-5767-418e-bd49-0a0b6377f35b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:05.004383 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa03dc77-5767-418e-bd49-0a0b6377f35b None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:05.024966 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-aa03dc77-5767-418e-bd49-0a0b6377f35b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:06:05.024966 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:05.024966 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:05.024966 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:05.024966 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:05.024966 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:05.024966 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:05.024966 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:05.024966 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:05.024966 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:05.024966 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:05.024966 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:05.024966 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:06:05.024966 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:06:05.024966 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:06:05.026118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:06:05.026118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:05.026118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:05.026118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:05.026118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:05.026118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:05.026118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:05.026118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:05.026118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:05.026118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:05.026118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:05.026118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:05.026118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:05.026118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:05.026118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:05.026118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:05.026118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:05.027235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:06:05.027235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:06:05.027235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:06:05.027235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:06:05.027235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:06:05.027235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:06:05.027235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:06:05.027235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:06:05.030865 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 602/1217] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:06:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:06:05.032069 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54b4b9-40f2-48c5-9d50-69d962604d1e None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:05.157695 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54b4b9-40f2-48c5-9d50-69d962604d1e None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:05.197934 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:05.202082 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9977c8-9d09-47ba-b4bf-e128fb4ee8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:05.203141 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9977c8-9d09-47ba-b4bf-e128fb4ee8a2 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:05.203626 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9977c8-9d09-47ba-b4bf-e128fb4ee8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:05.204088 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9977c8-9d09-47ba-b4bf-e128fb4ee8a2 None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:05.205160 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9977c8-9d09-47ba-b4bf-e128fb4ee8a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:05.206071 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9977c8-9d09-47ba-b4bf-e128fb4ee8a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:05.207777 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9977c8-9d09-47ba-b4bf-e128fb4ee8a2 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:05.224035 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 616/1218] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 27 21:06:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:05.231337 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03dc77-5767-418e-bd49-0a0b6377f35b None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:05.361999 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03dc77-5767-418e-bd49-0a0b6377f35b None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:05.396259 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:05.404729 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35422f6e-de55-42d9-986b-5233a597adaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:05.405426 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35422f6e-de55-42d9-986b-5233a597adaa None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:05.405708 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35422f6e-de55-42d9-986b-5233a597adaa None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:05.405982 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35422f6e-de55-42d9-986b-5233a597adaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:05.406826 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35422f6e-de55-42d9-986b-5233a597adaa None admin] RBAC: Adding query filter params (role.id=709aef6b85524eb2a8e9986a2bd9525b, scope.project.id=334438c929c44e489e9a3b93e6d37231, user.id=0437c45d72cd4a38b9d48ee6f50946fe) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:05.407122 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35422f6e-de55-42d9-986b-5233a597adaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:05.408596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35422f6e-de55-42d9-986b-5233a597adaa None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:05.416067 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 603/1219] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:06:05 2022] GET /identity/v3/role_assignments?user.id=0437c45d72cd4a38b9d48ee6f50946fe&scope.project.id=334438c929c44e489e9a3b93e6d37231&role.id=709aef6b85524eb2a8e9986a2bd9525b => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:07.594920 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c6966-eecd-4551-bf21-1a76f4804163 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:07.595739 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c6966-eecd-4551-bf21-1a76f4804163 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:07.595739 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c6966-eecd-4551-bf21-1a76f4804163 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:07.596918 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 617/1220] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:06:07.603080 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b1ee9-87db-4c83-b087-465136aace4e None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:07.603621 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b1ee9-87db-4c83-b087-465136aace4e None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:07.603621 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b1ee9-87db-4c83-b087-465136aace4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:07.621947 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-076b1ee9-87db-4c83-b087-465136aace4e None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:06:07.623260 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-076b1ee9-87db-4c83-b087-465136aace4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:06:07.704479 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 604/1221] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:06:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:06:07.731855 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c6966-eecd-4551-bf21-1a76f4804163 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:07.847962 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c6966-eecd-4551-bf21-1a76f4804163 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:07.889930 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc244686-e17a-4efd-89c7-c66c3962f9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:07.890797 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc244686-e17a-4efd-89c7-c66c3962f9f1 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:07.891127 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc244686-e17a-4efd-89c7-c66c3962f9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:07.891439 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc244686-e17a-4efd-89c7-c66c3962f9f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:07.904732 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc244686-e17a-4efd-89c7-c66c3962f9f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:06:07.913207 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-fc244686-e17a-4efd-89c7-c66c3962f9f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:06:07.913207 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:07.913207 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:07.913207 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:07.913207 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:07.913207 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:07.913207 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:07.913207 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:07.913207 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:07.913207 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:07.913207 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:07.913207 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:07.913207 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:06:07.913207 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:06:07.913207 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:06:07.913207 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:06:07.914487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:06:07.914487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:06:07.914487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:06:07.914487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:06:07.914487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:07.914487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:07.914487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:07.914487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:07.914487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:07.914487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:07.914487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:07.914487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:07.914487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:07.914487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:07.914487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:07.914487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:07.915653 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:07.915653 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:07.915653 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:07.915653 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:07.915653 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:06:07.915653 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:06:07.915653 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:06:07.915653 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:06:07.915653 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:06:07.915653 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:06:07.915653 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:06:07.915653 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:06:07.915653 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 618/1222] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 27 21:06:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:06:07.921429 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076b1ee9-87db-4c83-b087-465136aace4e None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:08.041801 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076b1ee9-87db-4c83-b087-465136aace4e None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:08.070719 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:08.084603 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dd742c-4f07-42ae-b673-44ffd9320484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:08.085475 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd742c-4f07-42ae-b673-44ffd9320484 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:08.085949 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd742c-4f07-42ae-b673-44ffd9320484 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:08.086313 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd742c-4f07-42ae-b673-44ffd9320484 None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:08.087256 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dd742c-4f07-42ae-b673-44ffd9320484 None admin] RBAC: Adding query filter params (name=service) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:08.087660 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dd742c-4f07-42ae-b673-44ffd9320484 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:08.089253 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dd742c-4f07-42ae-b673-44ffd9320484 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:08.099800 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 605/1223] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 27 21:06:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:08.106791 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc244686-e17a-4efd-89c7-c66c3962f9f1 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:08.162498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:08.232633 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc244686-e17a-4efd-89c7-c66c3962f9f1 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:08.276810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf001df-c36a-4ab7-9d8f-26feff909cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:08.277729 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf001df-c36a-4ab7-9d8f-26feff909cbc None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:08.278120 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf001df-c36a-4ab7-9d8f-26feff909cbc None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:08.278500 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf001df-c36a-4ab7-9d8f-26feff909cbc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:08.291703 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf001df-c36a-4ab7-9d8f-26feff909cbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:08.293313 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf001df-c36a-4ab7-9d8f-26feff909cbc None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:08.314879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-6bf001df-c36a-4ab7-9d8f-26feff909cbc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:08.314879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:08.314879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:08.314879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:08.314879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:08.314879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:08.314879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:08.314879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:08.314879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:08.314879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:08.314879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:08.314879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:08.314879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:06:08.314879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:06:08.314879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:06:08.316092 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:06:08.316092 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:08.316092 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:08.316092 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:08.316092 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:08.316092 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:08.316092 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:08.316092 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:08.316092 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:08.316092 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:08.316092 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:08.316092 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:08.316092 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:08.316092 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:08.316092 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:08.316092 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:08.316092 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:08.317393 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:06:08.317393 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:06:08.317393 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:08.317393 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:06:08.317393 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 619/1224] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:06:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:08.325324 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dd742c-4f07-42ae-b673-44ffd9320484 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:08.434070 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dd742c-4f07-42ae-b673-44ffd9320484 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:08.473555 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d71af5-5bdd-4004-a560-e353bf371a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:08.474915 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d71af5-5bdd-4004-a560-e353bf371a27 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:08.475360 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d71af5-5bdd-4004-a560-e353bf371a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:08.475755 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d71af5-5bdd-4004-a560-e353bf371a27 None admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:08.476825 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d71af5-5bdd-4004-a560-e353bf371a27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:08.477545 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d71af5-5bdd-4004-a560-e353bf371a27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:08.478626 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d71af5-5bdd-4004-a560-e353bf371a27 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:08.497377 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 606/1225] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:06:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:08.503164 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf001df-c36a-4ab7-9d8f-26feff909cbc None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:08.611438 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:08.619882 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf001df-c36a-4ab7-9d8f-26feff909cbc None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:08.657108 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe43ff3e-f24e-4b34-b749-53bc02336ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:08.658071 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe43ff3e-f24e-4b34-b749-53bc02336ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:08.658442 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe43ff3e-f24e-4b34-b749-53bc02336ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:08.658802 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe43ff3e-f24e-4b34-b749-53bc02336ae6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:08.668341 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe43ff3e-f24e-4b34-b749-53bc02336ae6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:08.670035 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe43ff3e-f24e-4b34-b749-53bc02336ae6 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:08.686770 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-fe43ff3e-f24e-4b34-b749-53bc02336ae6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 27 21:06:08.686770 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:08.686770 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:08.686770 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:08.686770 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:08.686770 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:08.686770 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:08.686770 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:08.686770 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:08.686770 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:08.686770 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:08.686770 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:08.686770 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:06:08.686770 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:06:08.686770 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:06:08.686770 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:06:08.687956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:08.687956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:08.687956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:06:08.687956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:06:08.687956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:06:08.687956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:06:08.687956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:08.687956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:08.687956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:08.687956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:08.687956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:08.687956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:08.687956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:08.687956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:08.687956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:08.687956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:08.687956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:08.687956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:08.689081 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:08.689081 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:08.689081 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:06:08.689081 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:06:08.689081 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:06:08.689081 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:06:08.689081 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:06:08.689081 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:06:08.689081 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 27 21:06:08.689081 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:06:08.689081 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 620/1226] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:06:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:08.694379 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d71af5-5bdd-4004-a560-e353bf371a27 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:08.755818 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:08.816131 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d71af5-5bdd-4004-a560-e353bf371a27 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:08.857991 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79356aa8-8ecc-47a4-9873-f1032dd89d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:08.858850 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79356aa8-8ecc-47a4-9873-f1032dd89d0c None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:08.859171 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79356aa8-8ecc-47a4-9873-f1032dd89d0c None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:08.859418 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79356aa8-8ecc-47a4-9873-f1032dd89d0c None admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:08.860303 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79356aa8-8ecc-47a4-9873-f1032dd89d0c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:08.860639 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79356aa8-8ecc-47a4-9873-f1032dd89d0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:08.862366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79356aa8-8ecc-47a4-9873-f1032dd89d0c None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:08.884613 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 607/1227] 10.222.0.13 () {56 vars in 1143 bytes} [Mon Jun 27 21:06:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:06:08.891133 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe43ff3e-f24e-4b34-b749-53bc02336ae6 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:09.007259 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe43ff3e-f24e-4b34-b749-53bc02336ae6 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:09.050031 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e5bfe1-81ad-41ce-950a-b036d63413b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:09.051302 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e5bfe1-81ad-41ce-950a-b036d63413b7 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:09.051748 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e5bfe1-81ad-41ce-950a-b036d63413b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:09.052236 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e5bfe1-81ad-41ce-950a-b036d63413b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:09.071343 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e5bfe1-81ad-41ce-950a-b036d63413b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:09.074021 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e5bfe1-81ad-41ce-950a-b036d63413b7 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:09.104247 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-f9e5bfe1-81ad-41ce-950a-b036d63413b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:09.104247 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:09.104247 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:09.104247 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:09.104247 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:09.104247 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:09.104247 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:09.104247 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:09.104247 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:09.104247 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:09.104247 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:09.104247 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:09.104247 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:06:09.104247 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:06:09.104247 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:06:09.105476 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:06:09.105476 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:09.105476 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:09.105476 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:09.105476 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:09.105476 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:09.105476 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:09.105476 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:09.105476 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:09.105476 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:09.105476 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:09.105476 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:09.105476 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:09.105476 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:09.105476 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:09.105476 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:09.105476 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:09.106484 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:06:09.106484 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:06:09.106484 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:09.106484 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:06:09.106484 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 621/1228] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:06:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:09.115863 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79356aa8-8ecc-47a4-9873-f1032dd89d0c None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:09.256276 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79356aa8-8ecc-47a4-9873-f1032dd89d0c None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:09.259433 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:09.294630 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a8aef-7f10-43ab-8b98-6dbdeca714e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:09.295387 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a8aef-7f10-43ab-8b98-6dbdeca714e5 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:09.295654 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a8aef-7f10-43ab-8b98-6dbdeca714e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:09.295963 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a8aef-7f10-43ab-8b98-6dbdeca714e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:09.296669 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54a8aef-7f10-43ab-8b98-6dbdeca714e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:09.297004 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54a8aef-7f10-43ab-8b98-6dbdeca714e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:09.298390 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54a8aef-7f10-43ab-8b98-6dbdeca714e5 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:09.310835 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 608/1229] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:06:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:09.316864 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e5bfe1-81ad-41ce-950a-b036d63413b7 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:09.328264 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:09.434428 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e5bfe1-81ad-41ce-950a-b036d63413b7 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:09.473014 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb569bbb-e23f-488c-a85e-735fb5773174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:09.474243 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb569bbb-e23f-488c-a85e-735fb5773174 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:09.474525 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb569bbb-e23f-488c-a85e-735fb5773174 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:09.474889 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb569bbb-e23f-488c-a85e-735fb5773174 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:09.483530 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb569bbb-e23f-488c-a85e-735fb5773174 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:09.485192 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb569bbb-e23f-488c-a85e-735fb5773174 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:09.502211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-bb569bbb-e23f-488c-a85e-735fb5773174 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:06:09.502211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:09.502211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:09.502211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:09.502211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:09.502211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:09.502211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:09.502211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:09.502211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:09.502211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:09.502211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:09.502211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:09.502211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:06:09.502211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:06:09.502211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:06:09.503564 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:06:09.503564 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:09.503564 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:09.503564 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:09.503564 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:09.503564 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:09.503564 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:09.503564 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:09.503564 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:09.503564 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:09.503564 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:09.503564 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:09.503564 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:09.503564 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:09.503564 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:09.503564 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:09.503564 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:09.504745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:06:09.504745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:06:09.504745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:06:09.504745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:06:09.504745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:06:09.504745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:06:09.504745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:06:09.504745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:06:09.504745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 622/1230] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:06:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:09.510146 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a8aef-7f10-43ab-8b98-6dbdeca714e5 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:09.627957 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a8aef-7f10-43ab-8b98-6dbdeca714e5 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:09.670528 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e3c09-9644-484b-9afa-9054f3eb9032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:09.671284 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e3c09-9644-484b-9afa-9054f3eb9032 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:09.671581 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e3c09-9644-484b-9afa-9054f3eb9032 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:09.671874 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e3c09-9644-484b-9afa-9054f3eb9032 None admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:09.672697 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e3c09-9644-484b-9afa-9054f3eb9032 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:09.673033 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e3c09-9644-484b-9afa-9054f3eb9032 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:09.674654 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e3c09-9644-484b-9afa-9054f3eb9032 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:09.688607 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 609/1231] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 27 21:06:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:09.695666 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb569bbb-e23f-488c-a85e-735fb5773174 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:09.777428 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:09.817532 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb569bbb-e23f-488c-a85e-735fb5773174 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:09.855622 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f393d-1561-4d6d-99cd-c2461b0eb99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:09.856670 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f393d-1561-4d6d-99cd-c2461b0eb99f None admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:09.857062 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f393d-1561-4d6d-99cd-c2461b0eb99f None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:09.857329 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f393d-1561-4d6d-99cd-c2461b0eb99f None admin] PATH_INFO: `/v3/projects/334438c929c44e489e9a3b93e6d37231/users/0437c45d72cd4a38b9d48ee6f50946fe/roles/709aef6b85524eb2a8e9986a2bd9525b` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:09.873987 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f393d-1561-4d6d-99cd-c2461b0eb99f None admin] RBAC: Authorizing `identity:create_grant(project_id=334438c929c44e489e9a3b93e6d37231, user_id=0437c45d72cd4a38b9d48ee6f50946fe, role_id=709aef6b85524eb2a8e9986a2bd9525b)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:09.875624 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f393d-1561-4d6d-99cd-c2461b0eb99f None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:09.889162 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 623/1232] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 27 21:06:09 2022] PUT /identity/v3/projects/334438c929c44e489e9a3b93e6d37231/users/0437c45d72cd4a38b9d48ee6f50946fe/roles/709aef6b85524eb2a8e9986a2bd9525b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:06:12.055225 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378fad6-0aa0-45ee-b7fe-3a471f8eaaff None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:12.055888 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378fad6-0aa0-45ee-b7fe-3a471f8eaaff None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:12.055888 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378fad6-0aa0-45ee-b7fe-3a471f8eaaff None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:12.057765 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 610/1233] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:06:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:06:12.063610 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c8499-e6b6-4877-a2d5-fbb013b8b3aa None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:12.064132 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c8499-e6b6-4877-a2d5-fbb013b8b3aa None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:12.064223 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4c8499-e6b6-4877-a2d5-fbb013b8b3aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:12.082561 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-ca4c8499-e6b6-4877-a2d5-fbb013b8b3aa None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:06:12.083837 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ca4c8499-e6b6-4877-a2d5-fbb013b8b3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:06:12.166686 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 624/1234] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:06:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:06:12.194648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3378fad6-0aa0-45ee-b7fe-3a471f8eaaff None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:12.225397 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:12.319501 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3378fad6-0aa0-45ee-b7fe-3a471f8eaaff None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:12.361599 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d11ef27-540f-4548-8a36-f784e112c06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:12.362625 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11ef27-540f-4548-8a36-f784e112c06c None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:12.362939 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11ef27-540f-4548-8a36-f784e112c06c None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:12.363260 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11ef27-540f-4548-8a36-f784e112c06c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:12.375793 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d11ef27-540f-4548-8a36-f784e112c06c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:06:12.384300 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-3d11ef27-540f-4548-8a36-f784e112c06c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:06:12.384300 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:12.384300 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:12.384300 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:12.384300 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:12.384300 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:12.384300 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:12.384300 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:12.384300 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:12.384300 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:12.384300 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:12.384300 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:12.384300 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:06:12.384300 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:06:12.384300 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:06:12.384300 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:06:12.385510 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:06:12.385510 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:06:12.385510 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:06:12.385510 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:06:12.385510 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:12.385510 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:12.385510 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:12.385510 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:12.385510 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:12.385510 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:12.385510 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:12.385510 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:12.385510 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:12.385510 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:12.385510 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:12.385510 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:12.386711 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:12.386711 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:12.386711 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:12.386711 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:12.386711 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:06:12.386711 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:06:12.386711 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:06:12.386711 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:06:12.386711 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:06:12.386711 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:06:12.386711 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 27 21:06:12.386711 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:06:12.386711 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 611/1235] 10.222.0.13 () {56 vars in 1102 bytes} [Mon Jun 27 21:06:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:12.391628 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4c8499-e6b6-4877-a2d5-fbb013b8b3aa None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:12.444543 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:12.509040 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4c8499-e6b6-4877-a2d5-fbb013b8b3aa None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:12.548335 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee053d-f113-4edc-9840-625037fad786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:12.549040 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee053d-f113-4edc-9840-625037fad786 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:12.549317 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee053d-f113-4edc-9840-625037fad786 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:12.549585 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee053d-f113-4edc-9840-625037fad786 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:12.550333 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ee053d-f113-4edc-9840-625037fad786 None admin] RBAC: Adding query filter params (name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:12.550631 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ee053d-f113-4edc-9840-625037fad786 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:12.552006 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ee053d-f113-4edc-9840-625037fad786 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:12.566503 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 625/1236] 10.222.0.13 () {56 vars in 1103 bytes} [Mon Jun 27 21:06:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:12.567827 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d11ef27-540f-4548-8a36-f784e112c06c None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:12.659132 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:12.677264 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d11ef27-540f-4548-8a36-f784e112c06c None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:12.712157 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b4c52-a622-4afb-aea8-afa355514082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:12.712991 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b4c52-a622-4afb-aea8-afa355514082 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:12.713276 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b4c52-a622-4afb-aea8-afa355514082 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:12.713549 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b4c52-a622-4afb-aea8-afa355514082 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:12.724880 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b4c52-a622-4afb-aea8-afa355514082 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:12.726744 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b4c52-a622-4afb-aea8-afa355514082 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:12.746260 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-932b4c52-a622-4afb-aea8-afa355514082 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:12.746260 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:12.746260 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:12.746260 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:12.746260 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:12.746260 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:12.746260 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:12.746260 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:12.746260 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:12.746260 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:12.746260 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:12.746260 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:12.746260 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:06:12.746260 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:06:12.746260 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:06:12.747286 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:06:12.747286 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:12.747286 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:12.747286 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:12.747286 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:12.747286 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:12.747286 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:12.747286 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:12.747286 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:12.747286 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:12.747286 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:12.747286 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:12.747286 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:12.747286 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:12.747286 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:12.747286 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:12.747286 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:12.748209 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:06:12.748209 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:06:12.748209 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:12.748209 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:06:12.748209 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 612/1237] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:06:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:12.753494 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee053d-f113-4edc-9840-625037fad786 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:12.874504 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee053d-f113-4edc-9840-625037fad786 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:12.884422 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:12.911189 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e0e63-bcbc-4257-97ce-a9e2fd95dd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:12.911815 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e0e63-bcbc-4257-97ce-a9e2fd95dd0a None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:12.912099 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e0e63-bcbc-4257-97ce-a9e2fd95dd0a None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:12.912372 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e0e63-bcbc-4257-97ce-a9e2fd95dd0a None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:12.913131 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067e0e63-bcbc-4257-97ce-a9e2fd95dd0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:12.913439 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067e0e63-bcbc-4257-97ce-a9e2fd95dd0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:12.914935 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067e0e63-bcbc-4257-97ce-a9e2fd95dd0a None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:12.932783 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 626/1238] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:12.934329 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b4c52-a622-4afb-aea8-afa355514082 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:13.054150 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b4c52-a622-4afb-aea8-afa355514082 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:13.093634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd43f3c1-6957-4ace-b42c-731a11ca7557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:13.094489 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43f3c1-6957-4ace-b42c-731a11ca7557 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:13.094741 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43f3c1-6957-4ace-b42c-731a11ca7557 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:13.095014 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd43f3c1-6957-4ace-b42c-731a11ca7557 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:13.098483 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:13.105183 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd43f3c1-6957-4ace-b42c-731a11ca7557 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:13.106721 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd43f3c1-6957-4ace-b42c-731a11ca7557 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:13.122955 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-cd43f3c1-6957-4ace-b42c-731a11ca7557 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 27 21:06:13.122955 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:13.122955 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:13.122955 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:13.122955 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:13.122955 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:13.122955 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:13.122955 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:13.122955 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:13.122955 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:13.122955 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:13.122955 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:13.122955 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:06:13.122955 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:06:13.122955 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:06:13.122955 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:06:13.124209 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:13.124209 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:13.124209 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:06:13.124209 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:06:13.124209 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:06:13.124209 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:06:13.124209 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:13.124209 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:13.124209 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:13.124209 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:13.124209 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:13.124209 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:13.124209 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:13.124209 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:13.124209 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:13.124209 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:13.124209 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:13.124209 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:13.125418 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:13.125418 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:13.125418 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:06:13.125418 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:06:13.125418 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:06:13.125418 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:06:13.125418 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:06:13.125418 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:06:13.125418 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 27 21:06:13.125418 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:06:13.125418 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 613/1239] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:06:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:13.130438 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e0e63-bcbc-4257-97ce-a9e2fd95dd0a None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:13.240756 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e0e63-bcbc-4257-97ce-a9e2fd95dd0a None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:13.282285 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3816acf6-5187-4458-ac5a-6d7d54192444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:13.283041 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816acf6-5187-4458-ac5a-6d7d54192444 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:13.283400 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816acf6-5187-4458-ac5a-6d7d54192444 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:13.283685 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816acf6-5187-4458-ac5a-6d7d54192444 None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:13.284661 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3816acf6-5187-4458-ac5a-6d7d54192444 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:13.285059 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3816acf6-5187-4458-ac5a-6d7d54192444 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:13.286642 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3816acf6-5187-4458-ac5a-6d7d54192444 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:13.307817 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 627/1240] 10.222.0.13 () {56 vars in 1143 bytes} [Mon Jun 27 21:06:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:13.314001 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd43f3c1-6957-4ace-b42c-731a11ca7557 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:13.428622 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd43f3c1-6957-4ace-b42c-731a11ca7557 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:13.467740 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15a5e6c-a835-45f4-8140-6690c35326dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:13.468407 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a5e6c-a835-45f4-8140-6690c35326dc None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:13.468687 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a5e6c-a835-45f4-8140-6690c35326dc None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:13.468992 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a5e6c-a835-45f4-8140-6690c35326dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:13.534927 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:13.543035 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15a5e6c-a835-45f4-8140-6690c35326dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:13.544588 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15a5e6c-a835-45f4-8140-6690c35326dc None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:13.659937 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 614/1241] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:06:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 21:06:13.666403 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3816acf6-5187-4458-ac5a-6d7d54192444 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:13.700709 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:13.790026 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3816acf6-5187-4458-ac5a-6d7d54192444 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:13.826278 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91aa26-baf9-412f-9b11-3b99fcb202a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:13.827173 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91aa26-baf9-412f-9b11-3b99fcb202a4 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:13.827519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91aa26-baf9-412f-9b11-3b99fcb202a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:13.827867 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91aa26-baf9-412f-9b11-3b99fcb202a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:13.839902 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91aa26-baf9-412f-9b11-3b99fcb202a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:13.842028 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91aa26-baf9-412f-9b11-3b99fcb202a4 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:13.861510 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-cc91aa26-baf9-412f-9b11-3b99fcb202a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:13.861510 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:13.861510 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:13.861510 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:13.861510 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:13.861510 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:13.861510 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:13.861510 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:13.861510 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:13.861510 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:13.861510 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:13.861510 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:13.861510 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:06:13.861510 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:06:13.861510 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:06:13.862958 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:06:13.862958 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:13.862958 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:13.862958 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:13.862958 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:13.862958 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:13.862958 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:13.862958 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:13.862958 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:13.862958 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:13.862958 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:13.862958 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:13.862958 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:13.862958 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:13.862958 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:13.862958 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:13.862958 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:13.864244 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:06:13.864244 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:06:13.864244 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:13.864244 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:06:13.864244 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 628/1242] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:06:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:13.868800 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15a5e6c-a835-45f4-8140-6690c35326dc None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:13.970906 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:13.978768 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15a5e6c-a835-45f4-8140-6690c35326dc None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:14.021655 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c980fa6-717f-4877-a3bd-f866ce49ac85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:14.022333 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c980fa6-717f-4877-a3bd-f866ce49ac85 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:14.022624 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c980fa6-717f-4877-a3bd-f866ce49ac85 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:14.022899 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c980fa6-717f-4877-a3bd-f866ce49ac85 None admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:14.023701 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c980fa6-717f-4877-a3bd-f866ce49ac85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:14.025021 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c980fa6-717f-4877-a3bd-f866ce49ac85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:14.025607 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c980fa6-717f-4877-a3bd-f866ce49ac85 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:14.037848 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 615/1243] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:06:14.043879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91aa26-baf9-412f-9b11-3b99fcb202a4 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:14.142051 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:14.167144 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91aa26-baf9-412f-9b11-3b99fcb202a4 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:14.204663 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f1c1f-4bad-44ed-a370-da1665475ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:14.205497 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f1c1f-4bad-44ed-a370-da1665475ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:14.205776 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f1c1f-4bad-44ed-a370-da1665475ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:14.206053 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f1c1f-4bad-44ed-a370-da1665475ff7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:14.214302 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035f1c1f-4bad-44ed-a370-da1665475ff7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:14.215941 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035f1c1f-4bad-44ed-a370-da1665475ff7 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:14.233813 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-035f1c1f-4bad-44ed-a370-da1665475ff7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:06:14.233813 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:14.233813 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:14.233813 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:14.233813 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:14.233813 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:14.233813 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:14.233813 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:14.233813 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:14.233813 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:14.233813 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:14.233813 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:14.233813 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:06:14.233813 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:06:14.233813 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:06:14.234805 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:06:14.234805 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:14.234805 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:14.234805 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:14.234805 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:14.234805 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:14.234805 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:14.234805 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:14.234805 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:14.234805 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:14.234805 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:14.234805 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:14.234805 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:14.234805 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:14.234805 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:14.234805 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:14.234805 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:14.236159 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:06:14.236159 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:06:14.236159 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:06:14.236159 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:06:14.236159 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:06:14.236159 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:06:14.236159 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:06:14.236159 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:06:14.236159 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 629/1244] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:06:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:14.243762 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c980fa6-717f-4877-a3bd-f866ce49ac85 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:14.353360 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c980fa6-717f-4877-a3bd-f866ce49ac85 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:14.396381 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fddedb-36f5-4b7c-b9f5-5c8cd79c8349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:14.397184 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fddedb-36f5-4b7c-b9f5-5c8cd79c8349 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:14.397472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fddedb-36f5-4b7c-b9f5-5c8cd79c8349 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:14.397749 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fddedb-36f5-4b7c-b9f5-5c8cd79c8349 None admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:14.398823 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fddedb-36f5-4b7c-b9f5-5c8cd79c8349 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:14.398999 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fddedb-36f5-4b7c-b9f5-5c8cd79c8349 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:14.400451 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fddedb-36f5-4b7c-b9f5-5c8cd79c8349 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:14.414200 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 616/1245] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 27 21:06:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:14.420514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f1c1f-4bad-44ed-a370-da1665475ff7 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:14.535097 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f1c1f-4bad-44ed-a370-da1665475ff7 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:14.576526 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75fd3c6-a74d-4d60-8532-679155c2cb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:14.577406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75fd3c6-a74d-4d60-8532-679155c2cb4e None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:14.577802 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75fd3c6-a74d-4d60-8532-679155c2cb4e None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:14.578142 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75fd3c6-a74d-4d60-8532-679155c2cb4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:14.579164 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75fd3c6-a74d-4d60-8532-679155c2cb4e None admin] RBAC: Adding query filter params (role.id=709aef6b85524eb2a8e9986a2bd9525b, scope.project.id=334438c929c44e489e9a3b93e6d37231, user.id=0437c45d72cd4a38b9d48ee6f50946fe) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:14.579514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75fd3c6-a74d-4d60-8532-679155c2cb4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:14.581222 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75fd3c6-a74d-4d60-8532-679155c2cb4e None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:14.590194 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 630/1246] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:06:14 2022] GET /identity/v3/role_assignments?user.id=0437c45d72cd4a38b9d48ee6f50946fe&scope.project.id=334438c929c44e489e9a3b93e6d37231&role.id=709aef6b85524eb2a8e9986a2bd9525b => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:16.780078 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59336c3-a9d0-4f37-9432-bae7a722785f None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:16.780780 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59336c3-a9d0-4f37-9432-bae7a722785f None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:16.780885 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59336c3-a9d0-4f37-9432-bae7a722785f None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:16.782438 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 617/1247] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:06:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:06:16.791446 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad9cd50-91cb-4e87-a703-c2e812cb3d48 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:16.792054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad9cd50-91cb-4e87-a703-c2e812cb3d48 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:16.792424 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad9cd50-91cb-4e87-a703-c2e812cb3d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:16.801380 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:16.817506 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-bad9cd50-91cb-4e87-a703-c2e812cb3d48 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:06:16.818741 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-bad9cd50-91cb-4e87-a703-c2e812cb3d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:06:16.897313 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 631/1248] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:06:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:06:16.922017 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59336c3-a9d0-4f37-9432-bae7a722785f None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:16.944049 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:17.047966 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59336c3-a9d0-4f37-9432-bae7a722785f None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:17.091083 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880dc62e-492b-4bd1-8d66-d5161022aa4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:17.092733 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880dc62e-492b-4bd1-8d66-d5161022aa4f None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:17.093218 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880dc62e-492b-4bd1-8d66-d5161022aa4f None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:17.093861 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880dc62e-492b-4bd1-8d66-d5161022aa4f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:17.107598 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880dc62e-492b-4bd1-8d66-d5161022aa4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:06:17.121297 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-880dc62e-492b-4bd1-8d66-d5161022aa4f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:06:17.121297 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:17.121297 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:17.121297 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:17.121297 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:17.121297 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:17.121297 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:17.121297 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:17.121297 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:17.121297 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:17.121297 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:17.121297 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:17.121297 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:06:17.121297 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:06:17.121297 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:06:17.121297 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:06:17.122516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:06:17.122516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:06:17.122516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:06:17.122516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:06:17.122516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:17.122516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:17.122516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:17.122516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:17.122516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:17.122516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:17.122516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:17.122516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:17.122516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:17.122516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:17.122516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:17.122516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:17.123451 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:17.123451 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:17.123451 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:17.123451 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:17.123451 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:06:17.123451 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:06:17.123451 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:06:17.123451 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:06:17.123451 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:06:17.123451 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:06:17.123451 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:06:17.123451 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:06:17.124149 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 618/1249] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 27 21:06:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:17.130229 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad9cd50-91cb-4e87-a703-c2e812cb3d48 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:17.265228 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad9cd50-91cb-4e87-a703-c2e812cb3d48 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:17.298795 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:17.304548 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d74130-7cf0-4061-9a53-5b4c33ea68a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:17.305378 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d74130-7cf0-4061-9a53-5b4c33ea68a9 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:17.305660 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d74130-7cf0-4061-9a53-5b4c33ea68a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:17.305968 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d74130-7cf0-4061-9a53-5b4c33ea68a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:17.306894 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d74130-7cf0-4061-9a53-5b4c33ea68a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:17.307226 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d74130-7cf0-4061-9a53-5b4c33ea68a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:17.308865 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d74130-7cf0-4061-9a53-5b4c33ea68a9 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:17.321739 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 632/1250] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:06:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:17.328810 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880dc62e-492b-4bd1-8d66-d5161022aa4f None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:17.433278 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:17.458366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880dc62e-492b-4bd1-8d66-d5161022aa4f None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:17.502546 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d8a9f6-2642-4969-95c9-49ba8a80839e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:17.503494 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8a9f6-2642-4969-95c9-49ba8a80839e None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:17.503794 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8a9f6-2642-4969-95c9-49ba8a80839e None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:17.504030 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8a9f6-2642-4969-95c9-49ba8a80839e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:17.517872 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d8a9f6-2642-4969-95c9-49ba8a80839e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:17.519522 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d8a9f6-2642-4969-95c9-49ba8a80839e None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:17.541530 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-d8d8a9f6-2642-4969-95c9-49ba8a80839e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:17.541530 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:17.541530 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:17.541530 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:17.541530 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:17.541530 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:17.541530 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:17.541530 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:17.541530 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:17.541530 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:17.541530 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:17.541530 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:17.541530 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:06:17.541530 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:06:17.541530 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:06:17.543418 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:06:17.543418 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:17.543418 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:17.543418 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:17.543418 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:17.543418 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:17.543418 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:17.543418 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:17.543418 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:17.543418 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:17.543418 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:17.543418 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:17.543418 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:17.543418 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:17.543418 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:17.543418 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:17.543418 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:17.545035 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:06:17.545035 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:06:17.545035 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:17.545035 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:06:17.545035 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 619/1251] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:06:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:17.549698 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d74130-7cf0-4061-9a53-5b4c33ea68a9 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:17.678333 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d74130-7cf0-4061-9a53-5b4c33ea68a9 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:17.719538 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55b986-3589-4886-a54a-caff59858155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:17.720217 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55b986-3589-4886-a54a-caff59858155 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:17.720495 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55b986-3589-4886-a54a-caff59858155 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:17.720854 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55b986-3589-4886-a54a-caff59858155 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:17.721534 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f55b986-3589-4886-a54a-caff59858155 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:17.721838 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f55b986-3589-4886-a54a-caff59858155 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:17.723223 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f55b986-3589-4886-a54a-caff59858155 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:17.737741 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 633/1252] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:06:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:17.746249 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d8a9f6-2642-4969-95c9-49ba8a80839e None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:17.902646 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d8a9f6-2642-4969-95c9-49ba8a80839e None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:17.935763 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:17.946439 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39267dad-0d9e-41a0-b752-dcfbcbf4c91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:17.947270 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39267dad-0d9e-41a0-b752-dcfbcbf4c91e None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:17.947555 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39267dad-0d9e-41a0-b752-dcfbcbf4c91e None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:17.947836 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39267dad-0d9e-41a0-b752-dcfbcbf4c91e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:17.957018 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39267dad-0d9e-41a0-b752-dcfbcbf4c91e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:17.958509 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39267dad-0d9e-41a0-b752-dcfbcbf4c91e None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:17.974333 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-39267dad-0d9e-41a0-b752-dcfbcbf4c91e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 27 21:06:17.974333 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:17.974333 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:17.974333 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:17.974333 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:17.974333 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:17.974333 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:17.974333 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:17.974333 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:17.974333 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:17.974333 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:17.974333 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:17.974333 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:06:17.974333 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:06:17.974333 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:06:17.974333 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:06:17.975693 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:17.975693 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:17.975693 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:06:17.975693 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:06:17.975693 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:06:17.975693 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:06:17.975693 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:17.975693 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:17.975693 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:17.975693 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:17.975693 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:17.975693 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:17.975693 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:17.975693 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:17.975693 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:17.975693 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:17.975693 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:17.975693 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:17.977270 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:17.977270 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:17.977270 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:06:17.977270 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:06:17.977270 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:06:17.977270 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:06:17.977270 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:06:17.977270 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:06:17.977270 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 27 21:06:17.977270 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:06:17.977270 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 620/1253] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:06:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:17.981840 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55b986-3589-4886-a54a-caff59858155 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:18.045228 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:18.096663 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55b986-3589-4886-a54a-caff59858155 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:18.134864 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbddec0-87ac-4c3a-aa83-a2e5eb84a234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:18.135548 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbddec0-87ac-4c3a-aa83-a2e5eb84a234 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:18.135829 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbddec0-87ac-4c3a-aa83-a2e5eb84a234 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:18.136111 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbddec0-87ac-4c3a-aa83-a2e5eb84a234 None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:18.136990 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbddec0-87ac-4c3a-aa83-a2e5eb84a234 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:18.137305 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbddec0-87ac-4c3a-aa83-a2e5eb84a234 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:18.138777 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbddec0-87ac-4c3a-aa83-a2e5eb84a234 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:18.159699 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 634/1254] 10.222.0.13 () {56 vars in 1143 bytes} [Mon Jun 27 21:06:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:18.165283 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39267dad-0d9e-41a0-b752-dcfbcbf4c91e None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:18.283293 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39267dad-0d9e-41a0-b752-dcfbcbf4c91e None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:18.329423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b664c9b-57d4-41e2-8e32-7648b5f16173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:18.330227 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b664c9b-57d4-41e2-8e32-7648b5f16173 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:18.330497 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b664c9b-57d4-41e2-8e32-7648b5f16173 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:18.330799 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b664c9b-57d4-41e2-8e32-7648b5f16173 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:18.404836 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:18.422538 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b664c9b-57d4-41e2-8e32-7648b5f16173 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:18.424037 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b664c9b-57d4-41e2-8e32-7648b5f16173 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:18.550679 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 621/1255] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:06:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 21:06:18.557490 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbddec0-87ac-4c3a-aa83-a2e5eb84a234 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:18.683266 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbddec0-87ac-4c3a-aa83-a2e5eb84a234 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:18.698216 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:18.723264 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7d4473-c41f-418e-b334-227b6df4e0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:18.724531 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d4473-c41f-418e-b334-227b6df4e0ab None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:18.724792 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d4473-c41f-418e-b334-227b6df4e0ab None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:18.725122 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d4473-c41f-418e-b334-227b6df4e0ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:18.738177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7d4473-c41f-418e-b334-227b6df4e0ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:18.739821 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7d4473-c41f-418e-b334-227b6df4e0ab None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:18.764282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-7e7d4473-c41f-418e-b334-227b6df4e0ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:18.764282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:18.764282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:18.764282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:18.764282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:18.764282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:18.764282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:18.764282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:18.764282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:18.764282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:18.764282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:18.764282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:18.764282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:06:18.764282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:06:18.764282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:06:18.765514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:06:18.765514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:18.765514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:18.765514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:18.765514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:18.765514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:18.765514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:18.765514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:18.765514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:18.765514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:18.765514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:18.765514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:18.765514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:18.765514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:18.765514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:18.765514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:18.765514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:18.766638 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:06:18.766638 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:06:18.766638 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:18.766638 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:06:18.766638 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 635/1256] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:06:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:18.772514 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b664c9b-57d4-41e2-8e32-7648b5f16173 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:18.877573 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:18.893466 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b664c9b-57d4-41e2-8e32-7648b5f16173 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:18.946005 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab23688-3f9e-4e37-9606-5573b3a3a3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:18.946669 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab23688-3f9e-4e37-9606-5573b3a3a3d2 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:18.946878 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab23688-3f9e-4e37-9606-5573b3a3a3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:18.947169 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab23688-3f9e-4e37-9606-5573b3a3a3d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:18.947982 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab23688-3f9e-4e37-9606-5573b3a3a3d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:18.948482 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab23688-3f9e-4e37-9606-5573b3a3a3d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:18.949891 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab23688-3f9e-4e37-9606-5573b3a3a3d2 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:18.963570 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 622/1257] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:18.969648 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7d4473-c41f-418e-b334-227b6df4e0ab None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:19.099266 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7d4473-c41f-418e-b334-227b6df4e0ab None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:19.144740 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad47ec3d-aa1e-4f75-be19-687e68976687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:19.145556 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad47ec3d-aa1e-4f75-be19-687e68976687 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:19.145835 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad47ec3d-aa1e-4f75-be19-687e68976687 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:19.146113 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad47ec3d-aa1e-4f75-be19-687e68976687 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:19.152867 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad47ec3d-aa1e-4f75-be19-687e68976687 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:19.154357 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad47ec3d-aa1e-4f75-be19-687e68976687 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:19.168145 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-ad47ec3d-aa1e-4f75-be19-687e68976687 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:06:19.168145 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:19.168145 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:19.168145 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:19.168145 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:19.168145 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:19.168145 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:19.168145 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:19.168145 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:19.168145 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:19.168145 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:19.168145 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:19.168145 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:06:19.168145 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:06:19.168145 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:06:19.169238 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:06:19.169238 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:19.169238 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:19.169238 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:19.169238 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:19.169238 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:19.169238 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:19.169238 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:19.169238 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:19.169238 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:19.169238 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:19.169238 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:19.169238 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:19.169238 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:19.169238 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:19.169238 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:19.169238 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:19.175188 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:06:19.175188 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:06:19.175188 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:06:19.175188 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:06:19.175188 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:06:19.175188 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:06:19.175188 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:06:19.175188 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:06:19.175188 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 636/1258] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:06:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:19.177128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab23688-3f9e-4e37-9606-5573b3a3a3d2 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:19.300317 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab23688-3f9e-4e37-9606-5573b3a3a3d2 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:19.341650 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827bb825-dc3f-4b23-b8f2-1f6d6b30df71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:19.342442 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827bb825-dc3f-4b23-b8f2-1f6d6b30df71 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:19.342852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827bb825-dc3f-4b23-b8f2-1f6d6b30df71 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:19.343086 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827bb825-dc3f-4b23-b8f2-1f6d6b30df71 None admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:19.344025 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827bb825-dc3f-4b23-b8f2-1f6d6b30df71 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:19.344368 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827bb825-dc3f-4b23-b8f2-1f6d6b30df71 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:19.346110 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827bb825-dc3f-4b23-b8f2-1f6d6b30df71 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:19.361602 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 623/1259] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 27 21:06:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:19.368762 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad47ec3d-aa1e-4f75-be19-687e68976687 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:19.398394 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:19.484998 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad47ec3d-aa1e-4f75-be19-687e68976687 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:19.526971 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccf46e3-f435-4914-a85f-e66154f4ab43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:19.527797 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf46e3-f435-4914-a85f-e66154f4ab43 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:19.528186 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf46e3-f435-4914-a85f-e66154f4ab43 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:19.528550 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccf46e3-f435-4914-a85f-e66154f4ab43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:19.529597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccf46e3-f435-4914-a85f-e66154f4ab43 None admin] RBAC: Adding query filter params (role.id=d336dd7b746c4488b33289c00302e4d5, scope.project.id=334438c929c44e489e9a3b93e6d37231, user.id=0437c45d72cd4a38b9d48ee6f50946fe) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:19.530080 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccf46e3-f435-4914-a85f-e66154f4ab43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:19.533245 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccf46e3-f435-4914-a85f-e66154f4ab43 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:19.540748 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 637/1260] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:06:19 2022] GET /identity/v3/role_assignments?user.id=0437c45d72cd4a38b9d48ee6f50946fe&scope.project.id=334438c929c44e489e9a3b93e6d37231&role.id=d336dd7b746c4488b33289c00302e4d5 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:06:21.657519 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e56816-5853-4b38-9cf9-72c804a160e2 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:21.658103 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e56816-5853-4b38-9cf9-72c804a160e2 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:21.658103 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e56816-5853-4b38-9cf9-72c804a160e2 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:21.659574 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 624/1261] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:06:21.665234 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa8c92-366a-4ab3-ab7c-b6f70b8466f6 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:21.665703 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa8c92-366a-4ab3-ab7c-b6f70b8466f6 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:21.665803 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa8c92-366a-4ab3-ab7c-b6f70b8466f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:21.683156 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-cdfa8c92-366a-4ab3-ab7c-b6f70b8466f6 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:06:21.684392 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-cdfa8c92-366a-4ab3-ab7c-b6f70b8466f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:06:21.772443 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 638/1262] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:06:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:06:21.799181 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e56816-5853-4b38-9cf9-72c804a160e2 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:21.811464 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:21.909030 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e56816-5853-4b38-9cf9-72c804a160e2 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:21.945805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff342735-5dcb-4b10-b5bf-0f98d01e0f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:21.946592 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff342735-5dcb-4b10-b5bf-0f98d01e0f81 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:21.946875 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff342735-5dcb-4b10-b5bf-0f98d01e0f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:21.947145 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff342735-5dcb-4b10-b5bf-0f98d01e0f81 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:21.957651 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff342735-5dcb-4b10-b5bf-0f98d01e0f81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:06:21.965379 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-ff342735-5dcb-4b10-b5bf-0f98d01e0f81 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:06:21.965379 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:21.965379 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:21.965379 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:21.965379 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:21.965379 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:21.965379 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:21.965379 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:21.965379 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:21.965379 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:21.965379 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:21.965379 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:21.965379 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:06:21.965379 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:06:21.965379 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:06:21.965379 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:06:21.966504 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:06:21.966504 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:06:21.966504 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:06:21.966504 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:06:21.966504 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:21.966504 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:21.966504 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:21.966504 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:21.966504 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:21.966504 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:21.966504 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:21.966504 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:21.966504 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:21.966504 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:21.966504 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:21.966504 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:21.967642 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:21.967642 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:21.967642 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:21.967642 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:21.967642 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:06:21.967642 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:06:21.967642 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:06:21.967642 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:06:21.967642 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:06:21.967642 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:06:21.967642 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:06:21.967642 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:06:21.967642 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 625/1263] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 27 21:06:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:21.973023 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa8c92-366a-4ab3-ab7c-b6f70b8466f6 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:21.995055 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:22.086120 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa8c92-366a-4ab3-ab7c-b6f70b8466f6 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:22.134290 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d37c9-db6d-487b-bb8f-4c5c21962f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:22.134956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d37c9-db6d-487b-bb8f-4c5c21962f46 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:22.135235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d37c9-db6d-487b-bb8f-4c5c21962f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:22.135534 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d37c9-db6d-487b-bb8f-4c5c21962f46 None admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:22.136332 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6d37c9-db6d-487b-bb8f-4c5c21962f46 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:22.136642 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6d37c9-db6d-487b-bb8f-4c5c21962f46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:22.138145 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6d37c9-db6d-487b-bb8f-4c5c21962f46 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:22.147181 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 639/1264] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:06:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:22.155221 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff342735-5dcb-4b10-b5bf-0f98d01e0f81 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:22.233785 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:22.271254 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff342735-5dcb-4b10-b5bf-0f98d01e0f81 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:22.306128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202f908f-7857-480d-ae07-7ea2d7c2127a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:22.306927 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f908f-7857-480d-ae07-7ea2d7c2127a None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:22.307208 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f908f-7857-480d-ae07-7ea2d7c2127a None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:22.307476 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f908f-7857-480d-ae07-7ea2d7c2127a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:22.318121 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202f908f-7857-480d-ae07-7ea2d7c2127a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:22.319572 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202f908f-7857-480d-ae07-7ea2d7c2127a None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:22.336440 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-202f908f-7857-480d-ae07-7ea2d7c2127a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:22.336440 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:22.336440 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:22.336440 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:22.336440 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:22.336440 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:22.336440 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:22.336440 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:22.336440 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:22.336440 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:22.336440 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:22.336440 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:22.336440 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:06:22.336440 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:06:22.336440 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:06:22.337552 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:06:22.337552 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:22.337552 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:22.337552 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:22.337552 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:22.337552 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:22.337552 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:22.337552 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:22.337552 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:22.337552 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:22.337552 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:22.337552 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:22.337552 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:22.337552 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:22.337552 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:22.337552 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:22.337552 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:22.338676 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:06:22.338676 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:06:22.338676 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:22.338676 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:06:22.338676 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 626/1265] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:06:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:22.343664 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d37c9-db6d-487b-bb8f-4c5c21962f46 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:22.430654 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:22.451171 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d37c9-db6d-487b-bb8f-4c5c21962f46 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:22.486591 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aac5c0-b8de-4125-aa8a-55562f51f136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:22.487174 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aac5c0-b8de-4125-aa8a-55562f51f136 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:22.487459 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aac5c0-b8de-4125-aa8a-55562f51f136 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:22.487736 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aac5c0-b8de-4125-aa8a-55562f51f136 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:22.488470 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aac5c0-b8de-4125-aa8a-55562f51f136 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:22.488779 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aac5c0-b8de-4125-aa8a-55562f51f136 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:22.490200 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aac5c0-b8de-4125-aa8a-55562f51f136 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:22.503061 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 640/1266] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:22.509474 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202f908f-7857-480d-ae07-7ea2d7c2127a None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:22.619568 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202f908f-7857-480d-ae07-7ea2d7c2127a None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:22.639798 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:22.654983 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17904ed2-427e-4c0a-8e9a-2fb37755f2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:22.655791 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17904ed2-427e-4c0a-8e9a-2fb37755f2f0 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:22.656074 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17904ed2-427e-4c0a-8e9a-2fb37755f2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:22.656347 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17904ed2-427e-4c0a-8e9a-2fb37755f2f0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:22.664423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17904ed2-427e-4c0a-8e9a-2fb37755f2f0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:22.666371 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17904ed2-427e-4c0a-8e9a-2fb37755f2f0 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:22.689962 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-17904ed2-427e-4c0a-8e9a-2fb37755f2f0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 27 21:06:22.689962 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:22.689962 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:22.689962 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:22.689962 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:22.689962 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:22.689962 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:22.689962 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:22.689962 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:22.689962 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:22.689962 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:22.689962 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:22.689962 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:06:22.689962 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:06:22.689962 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:06:22.689962 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:06:22.691137 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:22.691137 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:22.691137 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:06:22.691137 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:06:22.691137 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:06:22.691137 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:06:22.691137 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:22.691137 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:22.691137 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:22.691137 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:22.691137 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:22.691137 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:22.691137 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:22.691137 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:22.691137 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:22.691137 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:22.691137 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:22.691137 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:22.697356 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:22.697356 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:22.697356 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:06:22.697356 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:06:22.697356 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:06:22.697356 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:06:22.697356 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:06:22.697356 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:06:22.697356 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 27 21:06:22.697356 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:06:22.697356 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 627/1267] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:06:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:22.698103 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aac5c0-b8de-4125-aa8a-55562f51f136 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:22.817739 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aac5c0-b8de-4125-aa8a-55562f51f136 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:22.851450 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b0784-d9c4-4182-8253-a12e95e2160e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:22.852110 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b0784-d9c4-4182-8253-a12e95e2160e None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:22.852385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b0784-d9c4-4182-8253-a12e95e2160e None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:22.852656 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b0784-d9c4-4182-8253-a12e95e2160e None admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:22.853514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346b0784-d9c4-4182-8253-a12e95e2160e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:22.853825 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346b0784-d9c4-4182-8253-a12e95e2160e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:22.855249 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346b0784-d9c4-4182-8253-a12e95e2160e None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:22.865068 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:22.874900 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 641/1268] 10.222.0.13 () {56 vars in 1143 bytes} [Mon Jun 27 21:06:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:22.880732 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17904ed2-427e-4c0a-8e9a-2fb37755f2f0 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:22.982041 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17904ed2-427e-4c0a-8e9a-2fb37755f2f0 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:23.017544 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44191daf-0292-47a1-9967-0fb8b19d3141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:23.018380 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44191daf-0292-47a1-9967-0fb8b19d3141 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:23.018620 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44191daf-0292-47a1-9967-0fb8b19d3141 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:23.018895 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44191daf-0292-47a1-9967-0fb8b19d3141 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:23.031181 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44191daf-0292-47a1-9967-0fb8b19d3141 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:23.032628 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44191daf-0292-47a1-9967-0fb8b19d3141 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:23.050715 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-44191daf-0292-47a1-9967-0fb8b19d3141 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:23.050715 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:23.050715 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:23.050715 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:23.050715 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:23.050715 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:23.050715 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:23.050715 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:23.050715 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:23.050715 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:23.050715 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:23.050715 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:23.050715 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:06:23.050715 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:06:23.050715 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:06:23.051815 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:06:23.051815 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:23.051815 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:23.051815 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:23.051815 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:23.051815 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:23.051815 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:23.051815 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:23.051815 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:23.051815 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:23.051815 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:23.051815 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:23.051815 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:23.051815 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:23.051815 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:23.051815 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:23.051815 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:23.052728 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:06:23.052728 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:06:23.052728 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:23.052728 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:06:23.052728 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 628/1269] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:06:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 27 21:06:23.058199 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b0784-d9c4-4182-8253-a12e95e2160e None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:23.163919 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b0784-d9c4-4182-8253-a12e95e2160e None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:23.199237 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9995e6d-4034-4260-9124-8710fba40567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:23.199894 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9995e6d-4034-4260-9124-8710fba40567 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:23.200175 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9995e6d-4034-4260-9124-8710fba40567 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:23.200446 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9995e6d-4034-4260-9124-8710fba40567 None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:23.201203 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9995e6d-4034-4260-9124-8710fba40567 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:23.201511 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9995e6d-4034-4260-9124-8710fba40567 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:23.202888 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9995e6d-4034-4260-9124-8710fba40567 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:23.214211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 642/1270] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:23.220442 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44191daf-0292-47a1-9967-0fb8b19d3141 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:23.255456 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:23.325076 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44191daf-0292-47a1-9967-0fb8b19d3141 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:23.359431 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5492267a-eb23-44af-883f-aa45ab386c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:23.360257 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5492267a-eb23-44af-883f-aa45ab386c58 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:23.360519 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5492267a-eb23-44af-883f-aa45ab386c58 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:23.360799 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5492267a-eb23-44af-883f-aa45ab386c58 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:23.367910 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5492267a-eb23-44af-883f-aa45ab386c58 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:23.369399 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5492267a-eb23-44af-883f-aa45ab386c58 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:23.384198 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-5492267a-eb23-44af-883f-aa45ab386c58 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:06:23.384198 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:23.384198 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:23.384198 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:23.384198 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:23.384198 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:23.384198 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:23.384198 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:23.384198 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:23.384198 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:23.384198 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:23.384198 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:23.384198 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:06:23.384198 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:06:23.384198 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:06:23.385454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:06:23.385454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:23.385454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:23.385454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:23.385454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:23.385454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:23.385454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:23.385454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:23.385454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:23.385454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:23.385454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:23.385454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:23.385454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:23.385454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:23.385454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:23.385454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:23.385454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:23.386566 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:06:23.386566 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:06:23.386566 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:06:23.386566 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:06:23.386566 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:06:23.386566 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:06:23.386566 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:06:23.386566 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:06:23.386566 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 629/1271] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:06:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:23.392452 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9995e6d-4034-4260-9124-8710fba40567 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:23.455792 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:23.508956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9995e6d-4034-4260-9124-8710fba40567 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:23.547792 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2635d-de4b-4a1c-b9cd-6ce06f99380a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:23.548460 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2635d-de4b-4a1c-b9cd-6ce06f99380a None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:23.548765 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2635d-de4b-4a1c-b9cd-6ce06f99380a None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:23.549065 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2635d-de4b-4a1c-b9cd-6ce06f99380a None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:23.549861 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb2635d-de4b-4a1c-b9cd-6ce06f99380a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:23.550169 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb2635d-de4b-4a1c-b9cd-6ce06f99380a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:23.551611 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb2635d-de4b-4a1c-b9cd-6ce06f99380a None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:23.565337 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 643/1272] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 27 21:06:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:23.571582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5492267a-eb23-44af-883f-aa45ab386c58 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:23.678328 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:23.688359 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5492267a-eb23-44af-883f-aa45ab386c58 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:23.730334 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f667934-dbcb-4b62-afd8-a1279146f7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:23.731244 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f667934-dbcb-4b62-afd8-a1279146f7f1 None admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:23.731549 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f667934-dbcb-4b62-afd8-a1279146f7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:23.731843 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f667934-dbcb-4b62-afd8-a1279146f7f1 None admin] PATH_INFO: `/v3/projects/334438c929c44e489e9a3b93e6d37231/users/0437c45d72cd4a38b9d48ee6f50946fe/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:23.734547 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f667934-dbcb-4b62-afd8-a1279146f7f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=334438c929c44e489e9a3b93e6d37231, user_id=0437c45d72cd4a38b9d48ee6f50946fe, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:23.735994 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f667934-dbcb-4b62-afd8-a1279146f7f1 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:23.758043 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 630/1273] 10.222.0.13 () {58 vars in 1439 bytes} [Mon Jun 27 21:06:23 2022] PUT /identity/v3/projects/334438c929c44e489e9a3b93e6d37231/users/0437c45d72cd4a38b9d48ee6f50946fe/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:06:25.790745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df8274-af0d-4962-a98c-7ac3765809b5 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:25.791354 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df8274-af0d-4962-a98c-7ac3765809b5 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:25.791354 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df8274-af0d-4962-a98c-7ac3765809b5 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:25.792678 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 644/1274] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:06:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:06:25.798006 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399857c-47a3-42bf-8e83-efce9635fb31 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:25.798447 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399857c-47a3-42bf-8e83-efce9635fb31 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:25.798538 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399857c-47a3-42bf-8e83-efce9635fb31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:25.816093 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-a399857c-47a3-42bf-8e83-efce9635fb31 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:06:25.817372 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-a399857c-47a3-42bf-8e83-efce9635fb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:06:25.889263 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 631/1275] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:06:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:06:25.913324 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df8274-af0d-4962-a98c-7ac3765809b5 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:26.023001 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df8274-af0d-4962-a98c-7ac3765809b5 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:26.041715 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:26.060626 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de41f60-0bce-4643-a809-393f98ecf42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:26.061454 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de41f60-0bce-4643-a809-393f98ecf42d None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:26.061737 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de41f60-0bce-4643-a809-393f98ecf42d None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:26.062007 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de41f60-0bce-4643-a809-393f98ecf42d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:26.073165 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de41f60-0bce-4643-a809-393f98ecf42d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 27 21:06:26.081067 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-4de41f60-0bce-4643-a809-393f98ecf42d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:06:26.081067 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:26.081067 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:26.081067 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:26.081067 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:26.081067 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:26.081067 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:26.081067 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:26.081067 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:26.081067 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:26.081067 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:26.081067 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:26.081067 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 27 21:06:26.081067 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 27 21:06:26.081067 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 27 21:06:26.081067 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 27 21:06:26.082119 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 27 21:06:26.082119 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 27 21:06:26.082119 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 27 21:06:26.082119 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 27 21:06:26.082119 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:26.082119 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:26.082119 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:26.082119 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:26.082119 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:26.082119 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:26.082119 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:26.082119 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:26.082119 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:26.082119 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:26.082119 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:26.082119 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:26.083037 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:26.083037 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:26.083037 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:26.083037 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:26.083037 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 27 21:06:26.083037 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 27 21:06:26.083037 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 27 21:06:26.083037 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 27 21:06:26.083037 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 27 21:06:26.083037 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 27 21:06:26.083037 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 27 21:06:26.083037 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:06:26.083037 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 645/1276] 10.222.0.13 () {56 vars in 1096 bytes} [Mon Jun 27 21:06:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:26.088836 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399857c-47a3-42bf-8e83-efce9635fb31 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:26.178374 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:26.199509 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399857c-47a3-42bf-8e83-efce9635fb31 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:26.237029 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1e3ccd-5439-4080-8ded-30e9468fc874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:26.237705 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e3ccd-5439-4080-8ded-30e9468fc874 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:26.237990 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e3ccd-5439-4080-8ded-30e9468fc874 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:26.238269 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e3ccd-5439-4080-8ded-30e9468fc874 None admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:26.239016 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1e3ccd-5439-4080-8ded-30e9468fc874 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:26.239326 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1e3ccd-5439-4080-8ded-30e9468fc874 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:26.240705 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1e3ccd-5439-4080-8ded-30e9468fc874 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:26.249527 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 632/1277] 10.222.0.13 () {56 vars in 1099 bytes} [Mon Jun 27 21:06:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:26.255673 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de41f60-0bce-4643-a809-393f98ecf42d None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:26.362515 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de41f60-0bce-4643-a809-393f98ecf42d None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:26.399338 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c920c6d-af32-4bba-819b-fce287b6d7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:26.400134 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c920c6d-af32-4bba-819b-fce287b6d7ed None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:26.400419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c920c6d-af32-4bba-819b-fce287b6d7ed None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:26.400695 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c920c6d-af32-4bba-819b-fce287b6d7ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:26.411571 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c920c6d-af32-4bba-819b-fce287b6d7ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:26.413064 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c920c6d-af32-4bba-819b-fce287b6d7ed None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:26.430652 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-8c920c6d-af32-4bba-819b-fce287b6d7ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:26.430652 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:26.430652 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:26.430652 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:26.430652 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:26.430652 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:26.430652 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:26.430652 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:26.430652 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:26.430652 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:26.430652 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:26.430652 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:26.430652 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:06:26.430652 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:06:26.430652 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:06:26.431661 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:06:26.431661 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:26.431661 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:26.431661 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:26.431661 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:26.431661 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:26.431661 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:26.431661 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:26.431661 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:26.431661 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:26.431661 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:26.431661 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:26.431661 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:26.431661 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:26.431661 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:26.431661 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:26.431661 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:26.432740 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:06:26.432740 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:06:26.432740 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:26.432740 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:06:26.432740 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 646/1278] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:06:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:26.439268 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1e3ccd-5439-4080-8ded-30e9468fc874 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:26.539713 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1e3ccd-5439-4080-8ded-30e9468fc874 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:26.576481 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bcbc2d-28a6-4c85-b2e7-2270136d58e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:26.577175 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcbc2d-28a6-4c85-b2e7-2270136d58e5 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:26.577457 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcbc2d-28a6-4c85-b2e7-2270136d58e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:26.577731 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcbc2d-28a6-4c85-b2e7-2270136d58e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:26.578458 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bcbc2d-28a6-4c85-b2e7-2270136d58e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:26.578784 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bcbc2d-28a6-4c85-b2e7-2270136d58e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:26.580144 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bcbc2d-28a6-4c85-b2e7-2270136d58e5 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:26.592665 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 633/1279] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:06:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:26.599028 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c920c6d-af32-4bba-819b-fce287b6d7ed None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:26.633824 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:26.708325 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c920c6d-af32-4bba-819b-fce287b6d7ed None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:26.745115 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd99146-6356-4411-ba45-8a3125d20375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:26.745963 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd99146-6356-4411-ba45-8a3125d20375 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:26.746213 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd99146-6356-4411-ba45-8a3125d20375 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:26.746708 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd99146-6356-4411-ba45-8a3125d20375 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:26.756202 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd99146-6356-4411-ba45-8a3125d20375 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:26.757911 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd99146-6356-4411-ba45-8a3125d20375 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:26.779141 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-bdd99146-6356-4411-ba45-8a3125d20375 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 27 21:06:26.779141 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:26.779141 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:26.779141 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:26.779141 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:26.779141 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:26.779141 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:26.779141 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:26.779141 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:26.779141 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:26.779141 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:26.779141 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:26.779141 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 27 21:06:26.779141 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 27 21:06:26.779141 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 27 21:06:26.779141 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 27 21:06:26.780487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:26.780487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:26.780487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 27 21:06:26.780487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:06:26.780487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 27 21:06:26.780487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 27 21:06:26.780487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:26.780487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:26.780487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:26.780487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:26.780487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:26.780487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:26.780487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:26.780487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:26.780487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:26.780487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:26.780487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:26.780487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:26.781575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:26.781575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:26.781575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 27 21:06:26.781575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 27 21:06:26.781575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 27 21:06:26.781575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 27 21:06:26.781575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 27 21:06:26.781575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 27 21:06:26.781575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 27 21:06:26.781575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:06:26.781575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 647/1280] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:06:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:26.790796 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bcbc2d-28a6-4c85-b2e7-2270136d58e5 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:26.799106 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:26.904674 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bcbc2d-28a6-4c85-b2e7-2270136d58e5 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:26.938284 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c44298-af07-4e97-8379-f55344a3e8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:26.938958 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c44298-af07-4e97-8379-f55344a3e8ab None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:26.939241 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c44298-af07-4e97-8379-f55344a3e8ab None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:26.939516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c44298-af07-4e97-8379-f55344a3e8ab None admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:26.940353 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c44298-af07-4e97-8379-f55344a3e8ab None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:26.940666 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c44298-af07-4e97-8379-f55344a3e8ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:26.942140 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c44298-af07-4e97-8379-f55344a3e8ab None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:26.959602 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 634/1281] 10.222.0.13 () {56 vars in 1143 bytes} [Mon Jun 27 21:06:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:26.965830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd99146-6356-4411-ba45-8a3125d20375 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:27.064624 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:27.075814 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd99146-6356-4411-ba45-8a3125d20375 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:27.112678 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca120fd-4351-4535-a9c5-e00660b4b003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:27.113374 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca120fd-4351-4535-a9c5-e00660b4b003 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:27.113659 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca120fd-4351-4535-a9c5-e00660b4b003 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:27.113930 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca120fd-4351-4535-a9c5-e00660b4b003 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:27.192604 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca120fd-4351-4535-a9c5-e00660b4b003 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:27.194131 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca120fd-4351-4535-a9c5-e00660b4b003 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:27.296107 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:27.308327 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 648/1282] 10.222.0.13 () {58 vars in 1282 bytes} [Mon Jun 27 21:06:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 27 21:06:27.314584 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c44298-af07-4e97-8379-f55344a3e8ab None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:27.386290 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:27.430681 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c44298-af07-4e97-8379-f55344a3e8ab None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:27.467172 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce74550-8d3a-4bb8-8d69-1afbaf09707a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:27.467882 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce74550-8d3a-4bb8-8d69-1afbaf09707a None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:27.468163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce74550-8d3a-4bb8-8d69-1afbaf09707a None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:27.468432 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce74550-8d3a-4bb8-8d69-1afbaf09707a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:27.479828 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce74550-8d3a-4bb8-8d69-1afbaf09707a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:27.481319 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce74550-8d3a-4bb8-8d69-1afbaf09707a None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:27.500470 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-2ce74550-8d3a-4bb8-8d69-1afbaf09707a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:27.500470 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:27.500470 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:27.500470 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:27.500470 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:27.500470 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:27.500470 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:27.500470 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:27.500470 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:27.500470 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:27.500470 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:27.500470 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:27.500470 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 27 21:06:27.500470 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 27 21:06:27.500470 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 27 21:06:27.501579 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 27 21:06:27.501579 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:27.501579 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:27.501579 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:27.501579 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:27.501579 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:27.501579 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:27.501579 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:27.501579 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:27.501579 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:27.501579 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:27.501579 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:27.501579 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:27.501579 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:27.501579 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:27.501579 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:27.501579 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:27.502757 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 27 21:06:27.502757 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 27 21:06:27.502757 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 27 21:06:27.502757 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:06:27.502757 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 635/1283] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:06:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:27.507705 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca120fd-4351-4535-a9c5-e00660b4b003 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:27.614981 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca120fd-4351-4535-a9c5-e00660b4b003 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:27.652976 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c27a8-fdc1-4542-83ee-1c520dc423ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:27.653674 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c27a8-fdc1-4542-83ee-1c520dc423ce None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:27.653953 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c27a8-fdc1-4542-83ee-1c520dc423ce None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:27.654228 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c27a8-fdc1-4542-83ee-1c520dc423ce None admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:27.654967 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7c27a8-fdc1-4542-83ee-1c520dc423ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:27.655272 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7c27a8-fdc1-4542-83ee-1c520dc423ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:27.656699 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7c27a8-fdc1-4542-83ee-1c520dc423ce None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:27.670064 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 649/1284] 10.222.0.13 () {56 vars in 1109 bytes} [Mon Jun 27 21:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:27.675961 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce74550-8d3a-4bb8-8d69-1afbaf09707a None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:27.793448 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce74550-8d3a-4bb8-8d69-1afbaf09707a None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:27.808833 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:27.829534 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833071e1-9050-4c51-a12a-c34b28a2c424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:27.830344 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833071e1-9050-4c51-a12a-c34b28a2c424 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:27.830625 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833071e1-9050-4c51-a12a-c34b28a2c424 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:27.830895 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833071e1-9050-4c51-a12a-c34b28a2c424 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:27.837943 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833071e1-9050-4c51-a12a-c34b28a2c424 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:27.839390 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833071e1-9050-4c51-a12a-c34b28a2c424 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:27.854019 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application [None req-833071e1-9050-4c51-a12a-c34b28a2c424 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:06:27.854019 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:27.854019 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:27.854019 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:27.854019 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:27.854019 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:27.854019 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:27.854019 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:27.854019 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:27.854019 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:27.854019 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:27.854019 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:27.854019 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 27 21:06:27.854019 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 27 21:06:27.854019 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 27 21:06:27.855139 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 27 21:06:27.855139 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:27.855139 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:27.855139 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:27.855139 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:27.855139 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:27.855139 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:27.855139 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:27.855139 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:27.855139 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:27.855139 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:27.855139 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:27.855139 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:27.855139 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:27.855139 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:27.855139 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:27.855139 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:27.856267 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 27 21:06:27.856267 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 27 21:06:27.856267 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 27 21:06:27.856267 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 27 21:06:27.856267 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 27 21:06:27.856267 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 27 21:06:27.856267 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 27 21:06:27.856267 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: ERROR keystone.server.flask.application Jun 27 21:06:27.856267 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 636/1285] 10.222.0.13 () {56 vars in 1111 bytes} [Mon Jun 27 21:06:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:27.861185 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c27a8-fdc1-4542-83ee-1c520dc423ce None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:27.912917 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:27.963439 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c27a8-fdc1-4542-83ee-1c520dc423ce None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:27.997084 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd66829-b009-4d57-a2ae-b831275b2637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:27.997716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd66829-b009-4d57-a2ae-b831275b2637 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:27.997996 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd66829-b009-4d57-a2ae-b831275b2637 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:27.998269 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd66829-b009-4d57-a2ae-b831275b2637 None admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:27.999056 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd66829-b009-4d57-a2ae-b831275b2637 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:27.999362 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd66829-b009-4d57-a2ae-b831275b2637 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:28.000801 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd66829-b009-4d57-a2ae-b831275b2637 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:28.015265 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 650/1286] 10.222.0.13 () {56 vars in 1148 bytes} [Mon Jun 27 21:06:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:28.019166 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833071e1-9050-4c51-a12a-c34b28a2c424 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:28.141279 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833071e1-9050-4c51-a12a-c34b28a2c424 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:28.176497 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e913ba6-866d-49a9-918f-a6e87e92ae8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:28.177180 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e913ba6-866d-49a9-918f-a6e87e92ae8a None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:28.177463 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e913ba6-866d-49a9-918f-a6e87e92ae8a None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:28.177742 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e913ba6-866d-49a9-918f-a6e87e92ae8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:28.178579 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e913ba6-866d-49a9-918f-a6e87e92ae8a None admin] RBAC: Adding query filter params (role.id=d336dd7b746c4488b33289c00302e4d5, scope.project.id=334438c929c44e489e9a3b93e6d37231, user.id=0437c45d72cd4a38b9d48ee6f50946fe) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:28.178894 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e913ba6-866d-49a9-918f-a6e87e92ae8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:28.180391 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e913ba6-866d-49a9-918f-a6e87e92ae8a None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:28.186538 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 637/1287] 10.222.0.13 () {56 vars in 1374 bytes} [Mon Jun 27 21:06:28 2022] GET /identity/v3/role_assignments?user.id=0437c45d72cd4a38b9d48ee6f50946fe&scope.project.id=334438c929c44e489e9a3b93e6d37231&role.id=d336dd7b746c4488b33289c00302e4d5 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:30.255203 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c53a6a-90ab-49dc-b8d4-8148298144d2 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:30.255921 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c53a6a-90ab-49dc-b8d4-8148298144d2 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:30.255921 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c53a6a-90ab-49dc-b8d4-8148298144d2 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:30.257219 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 651/1288] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:06:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:06:30.262706 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c33345-d542-49b9-bbcf-505b188a7d97 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:30.263211 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c33345-d542-49b9-bbcf-505b188a7d97 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:30.263329 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c33345-d542-49b9-bbcf-505b188a7d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:30.283024 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-a2c33345-d542-49b9-bbcf-505b188a7d97 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:06:30.284121 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-a2c33345-d542-49b9-bbcf-505b188a7d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:06:30.315266 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:30.355236 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 638/1289] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:06:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:06:30.379534 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c53a6a-90ab-49dc-b8d4-8148298144d2 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:30.515053 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c53a6a-90ab-49dc-b8d4-8148298144d2 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:30.526666 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:30.551041 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7336da46-3ba1-4e73-ac96-f2b2a989ec7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:30.551775 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336da46-3ba1-4e73-ac96-f2b2a989ec7f None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:30.552054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336da46-3ba1-4e73-ac96-f2b2a989ec7f None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:30.552327 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336da46-3ba1-4e73-ac96-f2b2a989ec7f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:30.553006 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7336da46-3ba1-4e73-ac96-f2b2a989ec7f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:30.554502 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7336da46-3ba1-4e73-ac96-f2b2a989ec7f None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:30.566307 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-7336da46-3ba1-4e73-ac96-f2b2a989ec7f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 21:06:30.566307 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:30.566307 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:06:30.566307 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:06:30.566307 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:06:30.566307 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:06:30.566307 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:06:30.566307 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:06:30.566307 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 21:06:30.566307 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:06:30.566307 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:06:30.566307 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:06:30.566307 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:30.566307 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:30.566307 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:30.566307 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:30.566307 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:30.567522 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:30.567522 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:30.567522 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:30.567522 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:30.567522 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:30.567522 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:30.567522 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:06:30.567522 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:06:30.567522 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:06:30.567522 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:06:30.567522 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:30.567522 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:30.567522 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:30.567522 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:30.567522 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:30.567522 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:30.568557 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:30.568557 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:30.568557 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:30.568557 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:30.568557 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:30.568557 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:30.568557 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:30.568557 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:30.568557 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:30.568557 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:30.568557 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:06:30.568557 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:06:30.568557 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 21:06:30.568557 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:06:30.568557 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 652/1290] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Jun 27 21:06:30 2022] GET /identity/v3/services/placement => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:30.573730 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c33345-d542-49b9-bbcf-505b188a7d97 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:30.675393 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c33345-d542-49b9-bbcf-505b188a7d97 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:30.711432 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a15937-741e-402e-b0b4-fc9cd805b0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:30.712071 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a15937-741e-402e-b0b4-fc9cd805b0f8 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:30.712358 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a15937-741e-402e-b0b4-fc9cd805b0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:30.712630 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a15937-741e-402e-b0b4-fc9cd805b0f8 None admin] PATH_INFO: `/v3/services` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:30.713361 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a15937-741e-402e-b0b4-fc9cd805b0f8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:30.713663 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a15937-741e-402e-b0b4-fc9cd805b0f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:30.715027 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a15937-741e-402e-b0b4-fc9cd805b0f8 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:30.720014 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 639/1291] 10.222.0.13 () {56 vars in 1116 bytes} [Mon Jun 27 21:06:30 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:30.726728 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7336da46-3ba1-4e73-ac96-f2b2a989ec7f None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:30.847544 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7336da46-3ba1-4e73-ac96-f2b2a989ec7f None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:30.882539 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c3597f-40e6-41b6-af8c-27d58b58acf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:30.883201 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3597f-40e6-41b6-af8c-27d58b58acf0 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:30.883480 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3597f-40e6-41b6-af8c-27d58b58acf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:30.883753 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3597f-40e6-41b6-af8c-27d58b58acf0 None admin] PATH_INFO: `/v3/services` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:30.884465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c3597f-40e6-41b6-af8c-27d58b58acf0 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:30.884766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c3597f-40e6-41b6-af8c-27d58b58acf0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:30.886199 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c3597f-40e6-41b6-af8c-27d58b58acf0 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:30.890829 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 653/1292] 10.222.0.13 () {56 vars in 1116 bytes} [Mon Jun 27 21:06:30 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:32.999497 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c3957-3824-4ee1-91ff-1b85f5335871 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:33.000116 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c3957-3824-4ee1-91ff-1b85f5335871 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:33.000116 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c3957-3824-4ee1-91ff-1b85f5335871 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:33.002172 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 640/1293] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:06:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:06:33.012172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39dfd3c-759c-4f9d-926e-d31f06df9896 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:33.012639 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39dfd3c-759c-4f9d-926e-d31f06df9896 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:33.012905 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39dfd3c-759c-4f9d-926e-d31f06df9896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:33.036402 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-d39dfd3c-759c-4f9d-926e-d31f06df9896 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:06:33.037904 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d39dfd3c-759c-4f9d-926e-d31f06df9896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:06:33.071150 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:33.112119 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 654/1294] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:06:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:06:33.135562 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c3957-3824-4ee1-91ff-1b85f5335871 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:33.173472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:33.251989 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c3957-3824-4ee1-91ff-1b85f5335871 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:33.291448 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb473a91-a3da-47b4-a11f-f409dad009f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:33.292103 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb473a91-a3da-47b4-a11f-f409dad009f2 None admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:33.292377 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb473a91-a3da-47b4-a11f-f409dad009f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:33.292648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb473a91-a3da-47b4-a11f-f409dad009f2 None admin] PATH_INFO: `/v3/services` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:33.293419 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb473a91-a3da-47b4-a11f-f409dad009f2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:33.294779 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb473a91-a3da-47b4-a11f-f409dad009f2 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:33.310497 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 641/1295] 10.222.0.13 () {60 vars in 1141 bytes} [Mon Jun 27 21:06:33 2022] POST /identity/v3/services => generated 246 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:06:35.349235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b93db-1ee0-4b15-ab92-4c1db9a09838 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:35.349764 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b93db-1ee0-4b15-ab92-4c1db9a09838 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:35.349764 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b93db-1ee0-4b15-ab92-4c1db9a09838 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:35.351577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 655/1296] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:06:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:06:35.357971 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93daae62-762e-49a9-9e69-5591b1dfaa7c None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:35.358369 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93daae62-762e-49a9-9e69-5591b1dfaa7c None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:35.358561 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93daae62-762e-49a9-9e69-5591b1dfaa7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:35.384751 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-93daae62-762e-49a9-9e69-5591b1dfaa7c None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:06:35.385973 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-93daae62-762e-49a9-9e69-5591b1dfaa7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:06:35.464499 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 642/1297] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:06:35 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 27 21:06:35.489217 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b93db-1ee0-4b15-ab92-4c1db9a09838 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:35.606525 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b93db-1ee0-4b15-ab92-4c1db9a09838 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:35.643022 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26160a-b814-4905-a54a-908f59e0b357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:35.643827 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26160a-b814-4905-a54a-908f59e0b357 None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:35.644110 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26160a-b814-4905-a54a-908f59e0b357 None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:35.644381 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26160a-b814-4905-a54a-908f59e0b357 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:35.645071 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b26160a-b814-4905-a54a-908f59e0b357 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:35.646469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b26160a-b814-4905-a54a-908f59e0b357 None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:35.660614 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-4b26160a-b814-4905-a54a-908f59e0b357 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 21:06:35.660614 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:35.660614 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:06:35.660614 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:06:35.660614 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:06:35.660614 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:06:35.660614 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:06:35.660614 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:06:35.660614 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 21:06:35.660614 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:06:35.660614 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:06:35.660614 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:06:35.660614 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:35.660614 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:35.660614 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:35.660614 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:35.660614 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:35.662214 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:35.662214 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:35.662214 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:35.662214 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:35.662214 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:35.662214 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:35.662214 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:06:35.662214 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:06:35.662214 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:06:35.662214 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:06:35.662214 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:35.662214 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:35.662214 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:35.662214 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:35.662214 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:35.662214 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:35.663615 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:35.663615 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:35.663615 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:35.663615 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:35.663615 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:35.663615 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:35.663615 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:35.663615 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:35.663615 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:35.663615 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:35.663615 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:06:35.663615 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:06:35.663615 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 21:06:35.663615 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:06:35.663615 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 656/1298] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Jun 27 21:06:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:35.671446 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93daae62-762e-49a9-9e69-5591b1dfaa7c None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:35.702137 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:35.811639 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93daae62-762e-49a9-9e69-5591b1dfaa7c None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:35.853053 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa72d2-4591-44f9-a131-47aa11771679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:35.853726 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa72d2-4591-44f9-a131-47aa11771679 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:35.854012 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa72d2-4591-44f9-a131-47aa11771679 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:35.854277 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa72d2-4591-44f9-a131-47aa11771679 None admin] PATH_INFO: `/v3/services` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:35.854988 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa72d2-4591-44f9-a131-47aa11771679 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:35.855291 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa72d2-4591-44f9-a131-47aa11771679 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:35.856736 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa72d2-4591-44f9-a131-47aa11771679 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:35.862205 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 643/1299] 10.222.0.13 () {56 vars in 1116 bytes} [Mon Jun 27 21:06:35 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:06:35.869510 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26160a-b814-4905-a54a-908f59e0b357 None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:35.917125 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:35.995018 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26160a-b814-4905-a54a-908f59e0b357 None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:36.046307 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3547fde8-2c53-4f48-8ab8-41ed8853e2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:36.046969 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547fde8-2c53-4f48-8ab8-41ed8853e2de None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:36.047247 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547fde8-2c53-4f48-8ab8-41ed8853e2de None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:36.047514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547fde8-2c53-4f48-8ab8-41ed8853e2de None admin] PATH_INFO: `/v3/endpoints` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:36.048309 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3547fde8-2c53-4f48-8ab8-41ed8853e2de None admin] RBAC: Adding query filter params (interface=public, service_id=2079dc76ea284948a18967fcbf23e186, region_id=RegionOne) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:36.048617 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3547fde8-2c53-4f48-8ab8-41ed8853e2de None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:36.050027 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3547fde8-2c53-4f48-8ab8-41ed8853e2de None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:36.055880 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 657/1300] 10.222.0.13 () {56 vars in 1251 bytes} [Mon Jun 27 21:06:35 2022] GET /identity/v3/endpoints?service_id=2079dc76ea284948a18967fcbf23e186&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:06:36.061953 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa72d2-4591-44f9-a131-47aa11771679 None admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:36.149002 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:36.173147 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa72d2-4591-44f9-a131-47aa11771679 None admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:36.208170 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c011771-c888-4694-a76d-5866e758b42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:36.208827 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c011771-c888-4694-a76d-5866e758b42c None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:36.209143 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c011771-c888-4694-a76d-5866e758b42c None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:36.209417 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c011771-c888-4694-a76d-5866e758b42c None admin] PATH_INFO: `/v3/services` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:36.210105 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c011771-c888-4694-a76d-5866e758b42c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:36.211571 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c011771-c888-4694-a76d-5866e758b42c None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:36.218608 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 644/1301] 10.222.0.13 () {56 vars in 1087 bytes} [Mon Jun 27 21:06:36 2022] GET /identity/v3/services => generated 2191 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:06:38.228571 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c52c01-63fd-40ce-a4db-104e66015d34 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:38.229617 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c52c01-63fd-40ce-a4db-104e66015d34 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:38.229617 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c52c01-63fd-40ce-a4db-104e66015d34 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:38.231777 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 658/1302] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:06:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:06:38.239719 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0494296f-1021-49d3-9365-9220c6cab00f None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:38.240152 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0494296f-1021-49d3-9365-9220c6cab00f None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:38.240242 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0494296f-1021-49d3-9365-9220c6cab00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:38.259116 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-0494296f-1021-49d3-9365-9220c6cab00f None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:06:38.260291 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-0494296f-1021-49d3-9365-9220c6cab00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:06:38.331196 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 645/1303] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:06:38 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 27 21:06:38.354964 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c52c01-63fd-40ce-a4db-104e66015d34 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:38.454383 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:38.462005 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c52c01-63fd-40ce-a4db-104e66015d34 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:38.497530 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f324a-a7d1-4d06-9f7f-1ccb18f276ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:38.498325 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f324a-a7d1-4d06-9f7f-1ccb18f276ee None admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:38.498608 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f324a-a7d1-4d06-9f7f-1ccb18f276ee None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:38.498884 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f324a-a7d1-4d06-9f7f-1ccb18f276ee None admin] PATH_INFO: `/v3/services/placement` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:38.499532 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649f324a-a7d1-4d06-9f7f-1ccb18f276ee None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:38.500992 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649f324a-a7d1-4d06-9f7f-1ccb18f276ee None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:38.513319 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application [None req-649f324a-a7d1-4d06-9f7f-1ccb18f276ee None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 21:06:38.513319 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:38.513319 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 27 21:06:38.513319 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 27 21:06:38.513319 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 27 21:06:38.513319 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 27 21:06:38.513319 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 27 21:06:38.513319 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:06:38.513319 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 21:06:38.513319 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:06:38.513319 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 27 21:06:38.513319 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:06:38.513319 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 27 21:06:38.513319 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 27 21:06:38.513319 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 27 21:06:38.513319 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 27 21:06:38.513319 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 27 21:06:38.514635 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 27 21:06:38.514635 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 27 21:06:38.514635 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 27 21:06:38.514635 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 27 21:06:38.514635 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 27 21:06:38.514635 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 27 21:06:38.514635 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 27 21:06:38.514635 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 27 21:06:38.514635 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 27 21:06:38.514635 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 27 21:06:38.514635 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 27 21:06:38.514635 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 27 21:06:38.514635 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 27 21:06:38.514635 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 27 21:06:38.514635 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 27 21:06:38.514635 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.get_or_create( Jun 27 21:06:38.515922 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 27 21:06:38.515922 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application with Lock( Jun 27 21:06:38.515922 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 27 21:06:38.515922 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self._enter() Jun 27 21:06:38.515922 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 27 21:06:38.515922 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 27 21:06:38.515922 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 27 21:06:38.515922 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application return self.creator() Jun 27 21:06:38.515922 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 27 21:06:38.515922 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application created_value = creator( Jun 27 21:06:38.515922 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 27 21:06:38.515922 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 27 21:06:38.515922 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 27 21:06:38.515922 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: ERROR keystone.server.flask.application Jun 27 21:06:38.515922 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 659/1304] 10.222.0.13 () {56 vars in 1117 bytes} [Mon Jun 27 21:06:38 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 27 21:06:38.526382 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0494296f-1021-49d3-9365-9220c6cab00f None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:38.615553 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:38.644300 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0494296f-1021-49d3-9365-9220c6cab00f None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:38.681730 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd2151e-ac18-44c9-8741-9a325036c5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:38.682411 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2151e-ac18-44c9-8741-9a325036c5c3 None admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:38.682692 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2151e-ac18-44c9-8741-9a325036c5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:38.682969 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd2151e-ac18-44c9-8741-9a325036c5c3 None admin] PATH_INFO: `/v3/services` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:38.683695 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd2151e-ac18-44c9-8741-9a325036c5c3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:06:38.684003 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd2151e-ac18-44c9-8741-9a325036c5c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:38.685436 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd2151e-ac18-44c9-8741-9a325036c5c3 None admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:38.691775 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 646/1305] 10.222.0.13 () {56 vars in 1116 bytes} [Mon Jun 27 21:06:38 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:06:38.698417 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f324a-a7d1-4d06-9f7f-1ccb18f276ee None admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:06:38.823994 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f324a-a7d1-4d06-9f7f-1ccb18f276ee None admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:06:38.863588 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc823c-28eb-4000-b84d-2afab0d463ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:06:38.864240 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc823c-28eb-4000-b84d-2afab0d463ed None admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:06:38.864514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc823c-28eb-4000-b84d-2afab0d463ed None admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:06:38.864775 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc823c-28eb-4000-b84d-2afab0d463ed None admin] PATH_INFO: `/v3/endpoints` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:06:38.865537 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bc823c-28eb-4000-b84d-2afab0d463ed None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:06:38.867518 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bc823c-28eb-4000-b84d-2afab0d463ed None admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:06:38.888616 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 660/1306] 10.222.0.13 () {60 vars in 1144 bytes} [Mon Jun 27 21:06:38 2022] POST /identity/v3/endpoints => generated 331 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:06:48.688266 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=30 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:06:48.884173 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:07:43.500446 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1041d85-35c4-4c77-af42-45eeec98345a None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:07:43.501212 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1041d85-35c4-4c77-af42-45eeec98345a None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:07:43.501212 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1041d85-35c4-4c77-af42-45eeec98345a None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:07:43.510236 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 647/1307] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:07:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:07:43.521420 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d6209-aa5a-4043-b322-453bea8d9d4e None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:07:43.521420 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d6209-aa5a-4043-b322-453bea8d9d4e None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:07:43.521420 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d6209-aa5a-4043-b322-453bea8d9d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:07:43.660233 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-440d6209-aa5a-4043-b322-453bea8d9d4e None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:07:43.663748 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-440d6209-aa5a-4043-b322-453bea8d9d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:07:43.819629 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 661/1308] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:07:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:07:43.868992 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e7521-d079-4256-8c28-18940a87f506 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:07:43.869281 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e7521-d079-4256-8c28-18940a87f506 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:07:43.869559 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e7521-d079-4256-8c28-18940a87f506 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:07:43.871295 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 648/1309] 10.222.0.13 () {54 vars in 964 bytes} [Mon Jun 27 21:07:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:07:43.878090 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176c604-dcac-41c3-b259-038daf5eb004 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:07:43.879193 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176c604-dcac-41c3-b259-038daf5eb004 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:07:43.879482 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176c604-dcac-41c3-b259-038daf5eb004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:07:43.965496 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-c176c604-dcac-41c3-b259-038daf5eb004 None None] MFA Rules not processed for user `2a4e105858324391b5142d57e34aff9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:07:43.966760 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-c176c604-dcac-41c3-b259-038daf5eb004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:07:43.991857 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:07:44.059144 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 662/1310] 10.222.0.13 () {58 vars in 1061 bytes} [Mon Jun 27 21:07:43 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:07:44.068553 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e7521-d079-4256-8c28-18940a87f506 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:07:44.070658 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-274e7521-d079-4256-8c28-18940a87f506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:07:44.198246 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e7521-d079-4256-8c28-18940a87f506 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:07:44.281692 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f54b7-a4b1-446f-b167-4ac451c7e5e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:07:44.282357 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f54b7-a4b1-446f-b167-4ac451c7e5e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:07:44.282632 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f54b7-a4b1-446f-b167-4ac451c7e5e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:07:44.282960 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f54b7-a4b1-446f-b167-4ac451c7e5e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:07:44.284643 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-ba4f54b7-a4b1-446f-b167-4ac451c7e5e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:07:44.354766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:07:44.414440 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f54b7-a4b1-446f-b167-4ac451c7e5e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:07:44.415946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f54b7-a4b1-446f-b167-4ac451c7e5e9 service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:07:44.437000 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 649/1311] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:07:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:07:46.790994 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08289ad3-6083-499f-b11f-394350ecccff None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:07:46.791563 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08289ad3-6083-499f-b11f-394350ecccff None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:07:46.791563 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08289ad3-6083-499f-b11f-394350ecccff None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:07:46.792945 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 663/1312] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:07:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:07:46.798932 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6db2ff-9d05-4f15-8c5d-1e009da84a65 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:07:46.799374 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6db2ff-9d05-4f15-8c5d-1e009da84a65 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:07:46.799481 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6db2ff-9d05-4f15-8c5d-1e009da84a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:07:46.890010 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-5c6db2ff-9d05-4f15-8c5d-1e009da84a65 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:07:46.891294 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-5c6db2ff-9d05-4f15-8c5d-1e009da84a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:07:47.008598 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 650/1313] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:07:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:07:47.061304 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a7802-91c5-4ef2-8e6e-e52e7dfcb830 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:07:47.061587 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a7802-91c5-4ef2-8e6e-e52e7dfcb830 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:07:47.061932 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a7802-91c5-4ef2-8e6e-e52e7dfcb830 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:07:47.063472 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 664/1314] 10.222.0.13 () {54 vars in 964 bytes} [Mon Jun 27 21:07:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:07:47.069931 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892b4b6-ad79-4984-adda-641cee335f3a None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:07:47.070223 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892b4b6-ad79-4984-adda-641cee335f3a None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:07:47.070497 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892b4b6-ad79-4984-adda-641cee335f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:07:47.081521 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:07:47.156649 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-3892b4b6-ad79-4984-adda-641cee335f3a None None] MFA Rules not processed for user `2a4e105858324391b5142d57e34aff9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:07:47.157766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-3892b4b6-ad79-4984-adda-641cee335f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:07:47.239703 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 651/1315] 10.222.0.13 () {58 vars in 1061 bytes} [Mon Jun 27 21:07:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:07:47.249557 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64a7802-91c5-4ef2-8e6e-e52e7dfcb830 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:07:47.251438 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a64a7802-91c5-4ef2-8e6e-e52e7dfcb830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:07:47.362097 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64a7802-91c5-4ef2-8e6e-e52e7dfcb830 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:07:47.433992 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:07:47.436707 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5458a612-4994-4ed8-b105-17c6c4a74239 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:07:47.437406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458a612-4994-4ed8-b105-17c6c4a74239 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:07:47.437695 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458a612-4994-4ed8-b105-17c6c4a74239 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:07:47.437981 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458a612-4994-4ed8-b105-17c6c4a74239 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:07:47.440209 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-5458a612-4994-4ed8-b105-17c6c4a74239 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:07:47.553103 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5458a612-4994-4ed8-b105-17c6c4a74239 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:07:47.554572 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5458a612-4994-4ed8-b105-17c6c4a74239 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:07:47.572906 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 665/1316] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:07:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:07:50.168706 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eea188-9871-4096-98ba-3227d29a61f8 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:07:50.169388 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eea188-9871-4096-98ba-3227d29a61f8 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:07:50.169388 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eea188-9871-4096-98ba-3227d29a61f8 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:07:50.170690 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 652/1317] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:07:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:07:50.175956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c81e1-ed80-4601-b951-74e1e388b597 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:07:50.176419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c81e1-ed80-4601-b951-74e1e388b597 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:07:50.176525 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c81e1-ed80-4601-b951-74e1e388b597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:07:50.245369 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-731c81e1-ed80-4601-b951-74e1e388b597 None None] MFA Rules not processed for user `ed277620276442838d05c774bf48c920`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:07:50.246582 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-731c81e1-ed80-4601-b951-74e1e388b597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:07:50.302262 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:07:50.351183 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 666/1318] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:07:50 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:07:50.391213 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eea188-9871-4096-98ba-3227d29a61f8 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:07:50.392806 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-42eea188-9871-4096-98ba-3227d29a61f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:07:50.501918 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eea188-9871-4096-98ba-3227d29a61f8 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:07:50.504736 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:07:50.583058 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e0e4e-2e39-4529-a404-90e2126b6fd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:07:50.583612 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e0e4e-2e39-4529-a404-90e2126b6fd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:07:50.583888 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e0e4e-2e39-4529-a404-90e2126b6fd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:07:50.584345 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e0e4e-2e39-4529-a404-90e2126b6fd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:07:50.586565 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-099e0e4e-2e39-4529-a404-90e2126b6fd2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:07:50.711013 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e0e4e-2e39-4529-a404-90e2126b6fd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:07:50.712474 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e0e4e-2e39-4529-a404-90e2126b6fd2 service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:07:50.732422 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 653/1319] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:07:50 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:07:53.319283 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a6a45-1f9b-4a53-9831-b56d46f400a8 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:07:53.319808 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a6a45-1f9b-4a53-9831-b56d46f400a8 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:07:53.319808 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a6a45-1f9b-4a53-9831-b56d46f400a8 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:07:53.321127 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 667/1320] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:07:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:07:53.326758 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67011c16-15c6-4ef0-87bd-5ad791de7b7b None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:07:53.327238 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67011c16-15c6-4ef0-87bd-5ad791de7b7b None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:07:53.327343 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67011c16-15c6-4ef0-87bd-5ad791de7b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:07:53.368052 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:07:53.395355 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-67011c16-15c6-4ef0-87bd-5ad791de7b7b None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:07:53.396591 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-67011c16-15c6-4ef0-87bd-5ad791de7b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:07:53.500638 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 654/1321] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:07:53 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:07:53.539972 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85a6a45-1f9b-4a53-9831-b56d46f400a8 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:07:53.541705 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-c85a6a45-1f9b-4a53-9831-b56d46f400a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:07:53.665659 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85a6a45-1f9b-4a53-9831-b56d46f400a8 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:07:53.746091 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc116e-a881-4a63-9b19-f41a532b3fad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:07:53.746474 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc116e-a881-4a63-9b19-f41a532b3fad service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:07:53.746783 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc116e-a881-4a63-9b19-f41a532b3fad service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:07:53.747059 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc116e-a881-4a63-9b19-f41a532b3fad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:07:53.748756 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-c4cc116e-a881-4a63-9b19-f41a532b3fad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:07:53.757924 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:07:53.884065 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cc116e-a881-4a63-9b19-f41a532b3fad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:07:53.885546 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cc116e-a881-4a63-9b19-f41a532b3fad service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:07:53.905278 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 668/1322] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:07:53 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:07:56.885385 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179815b7-41e7-49f6-95f1-4ffc610e3038 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:07:56.885994 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179815b7-41e7-49f6-95f1-4ffc610e3038 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:07:56.885994 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179815b7-41e7-49f6-95f1-4ffc610e3038 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:07:56.887294 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 655/1323] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:07:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:07:56.893259 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac47344-e32f-4ea8-a1a5-38d1060cd637 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:07:56.893706 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac47344-e32f-4ea8-a1a5-38d1060cd637 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:07:56.893903 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac47344-e32f-4ea8-a1a5-38d1060cd637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:07:56.963375 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-dac47344-e32f-4ea8-a1a5-38d1060cd637 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:07:56.964617 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-dac47344-e32f-4ea8-a1a5-38d1060cd637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:07:57.028395 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:07:57.070988 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 669/1324] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:07:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:07:57.107497 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179815b7-41e7-49f6-95f1-4ffc610e3038 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:07:57.109448 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-179815b7-41e7-49f6-95f1-4ffc610e3038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:07:57.248702 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179815b7-41e7-49f6-95f1-4ffc610e3038 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:07:57.263482 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:07:57.332236 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3998c6d5-af08-42a6-9150-b5f84a4a35a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:07:57.332900 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998c6d5-af08-42a6-9150-b5f84a4a35a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:07:57.333212 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998c6d5-af08-42a6-9150-b5f84a4a35a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:07:57.333481 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3998c6d5-af08-42a6-9150-b5f84a4a35a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:07:57.335259 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-3998c6d5-af08-42a6-9150-b5f84a4a35a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:07:57.457817 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3998c6d5-af08-42a6-9150-b5f84a4a35a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:07:57.459401 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3998c6d5-af08-42a6-9150-b5f84a4a35a2 service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:07:57.478839 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 656/1325] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:07:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:07:59.637892 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb55fe-6537-4244-93c4-9db0c65dba55 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:07:59.638447 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb55fe-6537-4244-93c4-9db0c65dba55 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:07:59.638447 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb55fe-6537-4244-93c4-9db0c65dba55 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:07:59.639826 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 670/1326] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:07:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:07:59.645576 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90292aac-e007-4e8d-88cd-593ca7e06e2d None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:07:59.646052 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90292aac-e007-4e8d-88cd-593ca7e06e2d None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:07:59.646170 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90292aac-e007-4e8d-88cd-593ca7e06e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:07:59.699168 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:07:59.719238 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-90292aac-e007-4e8d-88cd-593ca7e06e2d None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:07:59.720447 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-90292aac-e007-4e8d-88cd-593ca7e06e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:07:59.821492 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 657/1327] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:07:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:07:59.857467 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb55fe-6537-4244-93c4-9db0c65dba55 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:07:59.859001 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ffeb55fe-6537-4244-93c4-9db0c65dba55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:07:59.970260 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb55fe-6537-4244-93c4-9db0c65dba55 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:08:00.056159 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75bbd3-c453-44c7-9d5b-b425cbf81d24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:08:00.056820 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75bbd3-c453-44c7-9d5b-b425cbf81d24 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:00.057140 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75bbd3-c453-44c7-9d5b-b425cbf81d24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:00.057508 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75bbd3-c453-44c7-9d5b-b425cbf81d24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:00.059170 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-aa75bbd3-c453-44c7-9d5b-b425cbf81d24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:00.077446 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:08:00.179208 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa75bbd3-c453-44c7-9d5b-b425cbf81d24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:08:00.180660 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa75bbd3-c453-44c7-9d5b-b425cbf81d24 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:08:00.207867 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 671/1328] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:07:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:08:02.287564 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ea80f-2382-440b-8100-baf1afa37d52 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:02.288272 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ea80f-2382-440b-8100-baf1afa37d52 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:02.288272 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ea80f-2382-440b-8100-baf1afa37d52 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:02.289519 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 658/1329] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:08:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:02.295200 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c5957-0fb2-47dc-b160-c2581ac3e02a None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:02.295685 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c5957-0fb2-47dc-b160-c2581ac3e02a None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:02.295781 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c5957-0fb2-47dc-b160-c2581ac3e02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:02.368074 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-7e4c5957-0fb2-47dc-b160-c2581ac3e02a None None] MFA Rules not processed for user `ed277620276442838d05c774bf48c920`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:02.369369 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-7e4c5957-0fb2-47dc-b160-c2581ac3e02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:02.440416 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:08:02.482524 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 672/1330] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:08:02 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:08:02.519821 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086ea80f-2382-440b-8100-baf1afa37d52 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:08:02.521454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-086ea80f-2382-440b-8100-baf1afa37d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:02.630184 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086ea80f-2382-440b-8100-baf1afa37d52 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:08:02.651276 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:08:02.708955 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260809e2-e032-4514-b407-6a3f0d724b00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:08:02.709680 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260809e2-e032-4514-b407-6a3f0d724b00 service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:02.709966 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260809e2-e032-4514-b407-6a3f0d724b00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:02.710235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260809e2-e032-4514-b407-6a3f0d724b00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:02.711889 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-260809e2-e032-4514-b407-6a3f0d724b00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:02.834599 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260809e2-e032-4514-b407-6a3f0d724b00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:08:02.836551 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260809e2-e032-4514-b407-6a3f0d724b00 service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:08:02.864728 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 659/1331] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:08:02 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:08:05.190718 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810322b-94b8-4748-bb44-edaaf8835d3d None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:05.191331 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810322b-94b8-4748-bb44-edaaf8835d3d None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:05.191331 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810322b-94b8-4748-bb44-edaaf8835d3d None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:05.192699 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 673/1332] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:08:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:05.199170 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950d2ea-5b3d-4428-a989-1cd2fdfe93b5 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:05.199717 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950d2ea-5b3d-4428-a989-1cd2fdfe93b5 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:05.199717 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950d2ea-5b3d-4428-a989-1cd2fdfe93b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:05.266565 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:08:05.269611 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-4950d2ea-5b3d-4428-a989-1cd2fdfe93b5 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:05.271272 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-4950d2ea-5b3d-4428-a989-1cd2fdfe93b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:05.377963 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 660/1333] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:08:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:08:05.415492 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f810322b-94b8-4748-bb44-edaaf8835d3d None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:08:05.417101 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-f810322b-94b8-4748-bb44-edaaf8835d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:05.531678 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f810322b-94b8-4748-bb44-edaaf8835d3d None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:08:05.611164 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3887a-46ff-4317-b3bd-d5bf8b35b173 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:08:05.611817 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3887a-46ff-4317-b3bd-d5bf8b35b173 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:05.612091 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3887a-46ff-4317-b3bd-d5bf8b35b173 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:05.612355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3887a-46ff-4317-b3bd-d5bf8b35b173 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:05.614047 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-64b3887a-46ff-4317-b3bd-d5bf8b35b173 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:05.643415 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:08:05.731958 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b3887a-46ff-4317-b3bd-d5bf8b35b173 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:08:05.733468 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b3887a-46ff-4317-b3bd-d5bf8b35b173 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:08:05.752852 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 674/1334] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:08:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:08:07.852161 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86601add-22d1-40c6-bae5-f464e1d89b42 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:07.852772 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86601add-22d1-40c6-bae5-f464e1d89b42 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:07.852772 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86601add-22d1-40c6-bae5-f464e1d89b42 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:07.854134 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 661/1335] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:08:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:07.859505 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e30921-1f4a-4fe9-a1c2-5d59e99fe4e4 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:07.860000 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e30921-1f4a-4fe9-a1c2-5d59e99fe4e4 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:07.860097 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e30921-1f4a-4fe9-a1c2-5d59e99fe4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:07.928312 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-00e30921-1f4a-4fe9-a1c2-5d59e99fe4e4 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:07.929599 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-00e30921-1f4a-4fe9-a1c2-5d59e99fe4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:08.008519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:08:08.037240 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 675/1336] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:08:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:08:08.077515 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86601add-22d1-40c6-bae5-f464e1d89b42 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:08:08.079055 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-86601add-22d1-40c6-bae5-f464e1d89b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:08.192068 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86601add-22d1-40c6-bae5-f464e1d89b42 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:08:08.223146 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:08:08.270597 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42578318-7453-480d-8b79-2fc8197ac835 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:08:08.271244 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42578318-7453-480d-8b79-2fc8197ac835 service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:08.271502 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42578318-7453-480d-8b79-2fc8197ac835 service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:08.272023 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42578318-7453-480d-8b79-2fc8197ac835 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:08.273491 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-42578318-7453-480d-8b79-2fc8197ac835 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:08.395797 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42578318-7453-480d-8b79-2fc8197ac835 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:08:08.397627 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42578318-7453-480d-8b79-2fc8197ac835 service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:08:08.417174 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 662/1337] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:08:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:08:10.870077 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f4db8-8920-4d40-b720-ee92b4d717bb None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:10.870687 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f4db8-8920-4d40-b720-ee92b4d717bb None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:10.870687 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f4db8-8920-4d40-b720-ee92b4d717bb None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:10.872324 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 676/1338] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:08:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:10.878179 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a2e76-80cf-4a68-879b-705c0e027620 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:10.878667 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a2e76-80cf-4a68-879b-705c0e027620 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:10.878772 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a2e76-80cf-4a68-879b-705c0e027620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:10.951443 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-db9a2e76-80cf-4a68-879b-705c0e027620 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:10.952704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-db9a2e76-80cf-4a68-879b-705c0e027620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:10.961696 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:08:11.058149 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 663/1339] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:08:10 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:08:11.094481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f4db8-8920-4d40-b720-ee92b4d717bb None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:08:11.096043 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-576f4db8-8920-4d40-b720-ee92b4d717bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:11.210158 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f4db8-8920-4d40-b720-ee92b4d717bb None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:08:11.285855 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d9f5a9-d9cc-41c7-986f-1445cc56ca21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:08:11.286511 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d9f5a9-d9cc-41c7-986f-1445cc56ca21 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:11.286789 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d9f5a9-d9cc-41c7-986f-1445cc56ca21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:11.287056 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d9f5a9-d9cc-41c7-986f-1445cc56ca21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:11.289520 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-84d9f5a9-d9cc-41c7-986f-1445cc56ca21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:11.328414 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:08:11.406904 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d9f5a9-d9cc-41c7-986f-1445cc56ca21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:08:11.408332 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d9f5a9-d9cc-41c7-986f-1445cc56ca21 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:08:11.427032 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 677/1340] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:08:11 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:08:15.629822 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f6094-582d-46ab-b45d-4b2c00d4c561 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:15.630379 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f6094-582d-46ab-b45d-4b2c00d4c561 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:15.630379 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f6094-582d-46ab-b45d-4b2c00d4c561 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:15.631739 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 664/1341] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:08:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:15.637341 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba007598-be56-4e0f-9fb3-fb3856a0b320 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:15.637863 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba007598-be56-4e0f-9fb3-fb3856a0b320 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:15.637863 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba007598-be56-4e0f-9fb3-fb3856a0b320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:15.717668 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-ba007598-be56-4e0f-9fb3-fb3856a0b320 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:15.719045 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ba007598-be56-4e0f-9fb3-fb3856a0b320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:15.818634 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:08:15.832806 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 678/1342] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:08:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:08:15.879627 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6f6094-582d-46ab-b45d-4b2c00d4c561 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:08:15.882445 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-eb6f6094-582d-46ab-b45d-4b2c00d4c561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:15.986891 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6f6094-582d-46ab-b45d-4b2c00d4c561 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:08:15.990916 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6619117f-c406-4e47-8e23-9c415b541f31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:08:15.991587 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619117f-c406-4e47-8e23-9c415b541f31 service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:15.991872 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619117f-c406-4e47-8e23-9c415b541f31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:15.992147 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619117f-c406-4e47-8e23-9c415b541f31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:16.002199 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6619117f-c406-4e47-8e23-9c415b541f31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:08:16.003650 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6619117f-c406-4e47-8e23-9c415b541f31 service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:08:16.014091 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 665/1343] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:08:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:08:18.686511 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571840f8-980d-415f-a645-22ac455099b9 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:18.687089 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571840f8-980d-415f-a645-22ac455099b9 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:18.687089 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571840f8-980d-415f-a645-22ac455099b9 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:18.688467 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 679/1344] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:08:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:18.694203 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b682519-a0c9-4ae1-862f-d3ee7401bb8b None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:18.694886 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b682519-a0c9-4ae1-862f-d3ee7401bb8b None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:18.694886 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b682519-a0c9-4ae1-862f-d3ee7401bb8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:18.726338 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-2b682519-a0c9-4ae1-862f-d3ee7401bb8b None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:18.727708 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-2b682519-a0c9-4ae1-862f-d3ee7401bb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:18.760847 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:08:18.815890 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 666/1345] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:08:18 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:08:18.854560 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571840f8-980d-415f-a645-22ac455099b9 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:08:18.856384 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-571840f8-980d-415f-a645-22ac455099b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:18.871892 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571840f8-980d-415f-a645-22ac455099b9 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:08:18.875553 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175db39e-3fa1-4ee9-bb8c-5ac3829ff429 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:08:18.876222 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175db39e-3fa1-4ee9-bb8c-5ac3829ff429 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:18.876503 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175db39e-3fa1-4ee9-bb8c-5ac3829ff429 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:18.876782 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175db39e-3fa1-4ee9-bb8c-5ac3829ff429 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:18.886915 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175db39e-3fa1-4ee9-bb8c-5ac3829ff429 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:08:18.888411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175db39e-3fa1-4ee9-bb8c-5ac3829ff429 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:08:18.898959 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 680/1346] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:08:18 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:08:22.905590 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252f3b8-c0a9-4643-b5fa-903560f90eb0 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:22.906176 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252f3b8-c0a9-4643-b5fa-903560f90eb0 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:22.906176 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252f3b8-c0a9-4643-b5fa-903560f90eb0 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:22.907424 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 667/1347] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:08:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:22.913020 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641b37d-9924-4845-a62e-3393167039d7 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:22.913526 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641b37d-9924-4845-a62e-3393167039d7 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:22.913613 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641b37d-9924-4845-a62e-3393167039d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:22.931213 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-b641b37d-9924-4845-a62e-3393167039d7 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:22.932399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-b641b37d-9924-4845-a62e-3393167039d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:22.940823 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 681/1348] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:08:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:08:22.979201 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6252f3b8-c0a9-4643-b5fa-903560f90eb0 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:08:22.990185 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6252f3b8-c0a9-4643-b5fa-903560f90eb0 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:08:22.994273 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d85cee-a482-470b-83b7-a23c090a1b77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:08:22.994928 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d85cee-a482-470b-83b7-a23c090a1b77 service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:22.995210 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d85cee-a482-470b-83b7-a23c090a1b77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:22.995524 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d85cee-a482-470b-83b7-a23c090a1b77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:23.005738 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d85cee-a482-470b-83b7-a23c090a1b77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:08:23.007213 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d85cee-a482-470b-83b7-a23c090a1b77 service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:08:23.018021 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 668/1349] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:08:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:08:28.134950 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2902614-3d61-4229-88c2-6b464afb4719 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:28.135810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2902614-3d61-4229-88c2-6b464afb4719 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:28.136252 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2902614-3d61-4229-88c2-6b464afb4719 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:28.137732 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 682/1350] 10.222.0.13 () {54 vars in 921 bytes} [Mon Jun 27 21:08:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:28.143756 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee70f5-4133-4458-b036-8c0f53104b64 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:28.144255 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee70f5-4133-4458-b036-8c0f53104b64 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:28.144476 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee70f5-4133-4458-b036-8c0f53104b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:28.248437 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-7fee70f5-4133-4458-b036-8c0f53104b64 None None] MFA Rules not processed for user `0437c45d72cd4a38b9d48ee6f50946fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:28.250257 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-7fee70f5-4133-4458-b036-8c0f53104b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:28.452317 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 669/1351] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Jun 27 21:08:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:08:32.766099 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a632b6-8256-400e-90a1-89a9a73572de None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:32.766684 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a632b6-8256-400e-90a1-89a9a73572de None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:32.766684 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a632b6-8256-400e-90a1-89a9a73572de None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:32.768238 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 683/1352] 10.222.0.13 () {54 vars in 921 bytes} [Mon Jun 27 21:08:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:32.773397 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d19875-ab53-4bec-863c-2da37a96c0c2 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:32.774318 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d19875-ab53-4bec-863c-2da37a96c0c2 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:32.774769 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d19875-ab53-4bec-863c-2da37a96c0c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:32.795132 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-22d19875-ab53-4bec-863c-2da37a96c0c2 None None] MFA Rules not processed for user `0437c45d72cd4a38b9d48ee6f50946fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:32.796817 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-22d19875-ab53-4bec-863c-2da37a96c0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:32.807145 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 670/1353] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Jun 27 21:08:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:08:32.925966 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=13 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:08:34.346769 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5d41d-c0db-40bb-a949-ebf279e2794b None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:34.347390 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5d41d-c0db-40bb-a949-ebf279e2794b None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:34.347390 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5d41d-c0db-40bb-a949-ebf279e2794b None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:34.348634 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 684/1354] 10.222.0.13 () {54 vars in 921 bytes} [Mon Jun 27 21:08:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:34.354289 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2773092d-68c1-44f4-9d7e-4fd2d90eadaf None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:34.354803 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2773092d-68c1-44f4-9d7e-4fd2d90eadaf None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:34.354897 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2773092d-68c1-44f4-9d7e-4fd2d90eadaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:34.376935 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-2773092d-68c1-44f4-9d7e-4fd2d90eadaf None None] MFA Rules not processed for user `0437c45d72cd4a38b9d48ee6f50946fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:34.378177 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-2773092d-68c1-44f4-9d7e-4fd2d90eadaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:34.387586 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 671/1355] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Jun 27 21:08:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:08:37.890404 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c46d81-eaa2-47c3-80d0-58f0e0122d42 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:37.890998 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c46d81-eaa2-47c3-80d0-58f0e0122d42 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:37.890998 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c46d81-eaa2-47c3-80d0-58f0e0122d42 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:37.892431 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 685/1356] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:08:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:37.902888 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3e63d-b75a-4769-9aed-9ac41514c5f2 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:37.903966 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3e63d-b75a-4769-9aed-9ac41514c5f2 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:37.904449 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3e63d-b75a-4769-9aed-9ac41514c5f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:37.931140 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-ceb3e63d-b75a-4769-9aed-9ac41514c5f2 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:37.933186 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-ceb3e63d-b75a-4769-9aed-9ac41514c5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:37.944012 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 672/1357] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:08:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:08:38.002975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec27f91-d046-434d-beb3-bf87250675c6 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:38.003491 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec27f91-d046-434d-beb3-bf87250675c6 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:38.003938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec27f91-d046-434d-beb3-bf87250675c6 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:38.006344 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 686/1358] 10.222.0.13 () {54 vars in 961 bytes} [Mon Jun 27 21:08:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:38.021070 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7f7ba-f0bc-4726-81c3-8239003ddf05 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:38.021573 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7f7ba-f0bc-4726-81c3-8239003ddf05 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:38.022101 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae7f7ba-f0bc-4726-81c3-8239003ddf05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:38.066340 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:08:38.077292 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-8ae7f7ba-f0bc-4726-81c3-8239003ddf05 None None] MFA Rules not processed for user `1366958034d540b4ba2feb421a6afabc`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:38.078480 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-8ae7f7ba-f0bc-4726-81c3-8239003ddf05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:38.164495 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 673/1359] 10.222.0.13 () {58 vars in 1058 bytes} [Mon Jun 27 21:08:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:08:38.172328 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec27f91-d046-434d-beb3-bf87250675c6 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:08:38.186794 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec27f91-d046-434d-beb3-bf87250675c6 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:08:38.192180 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a81d08c-e215-4066-b3f7-4b33bd274b34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:08:38.193654 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81d08c-e215-4066-b3f7-4b33bd274b34 service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:38.193924 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81d08c-e215-4066-b3f7-4b33bd274b34 service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:38.194407 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81d08c-e215-4066-b3f7-4b33bd274b34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:38.205533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a81d08c-e215-4066-b3f7-4b33bd274b34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:08:38.207131 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a81d08c-e215-4066-b3f7-4b33bd274b34 service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:08:38.217544 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 687/1360] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:08:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:08:40.505170 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42160a-ef69-4702-8ea3-6e72421733ff None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:40.505730 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42160a-ef69-4702-8ea3-6e72421733ff None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:40.505730 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42160a-ef69-4702-8ea3-6e72421733ff None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:40.507377 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 674/1361] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:08:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:40.512901 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91c86b-bfc4-4895-a4f9-bfc4ddc5694e None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:40.513745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91c86b-bfc4-4895-a4f9-bfc4ddc5694e None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:40.514120 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91c86b-bfc4-4895-a4f9-bfc4ddc5694e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:40.533427 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-4b91c86b-bfc4-4895-a4f9-bfc4ddc5694e None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:40.534933 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-4b91c86b-bfc4-4895-a4f9-bfc4ddc5694e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:40.543477 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 688/1362] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:08:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:08:40.587846 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42160a-ef69-4702-8ea3-6e72421733ff None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:08:40.606374 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42160a-ef69-4702-8ea3-6e72421733ff None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:08:40.613111 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef70fa0-fc09-4d74-972c-0330486dc2a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:08:40.614441 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef70fa0-fc09-4d74-972c-0330486dc2a0 service nova] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:40.615068 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef70fa0-fc09-4d74-972c-0330486dc2a0 service nova] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:40.615611 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef70fa0-fc09-4d74-972c-0330486dc2a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:40.626995 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef70fa0-fc09-4d74-972c-0330486dc2a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:08:40.629661 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef70fa0-fc09-4d74-972c-0330486dc2a0 service nova] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:08:40.641352 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 675/1363] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:08:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:08:43.030491 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17842fd-af7a-484c-9150-e23f58a12a6d None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:43.030491 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17842fd-af7a-484c-9150-e23f58a12a6d None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:43.030491 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17842fd-af7a-484c-9150-e23f58a12a6d None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:43.031858 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 689/1364] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:08:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:43.037903 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45870a-bf6e-4795-87d6-bc7b70ee7c18 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:43.038420 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45870a-bf6e-4795-87d6-bc7b70ee7c18 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:43.038420 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45870a-bf6e-4795-87d6-bc7b70ee7c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:43.058071 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-4f45870a-bf6e-4795-87d6-bc7b70ee7c18 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:43.059291 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-4f45870a-bf6e-4795-87d6-bc7b70ee7c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:43.068469 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 676/1365] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:08:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:08:43.118604 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88760b1-ee50-4da9-8061-e20f55d53fbb None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:43.118957 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88760b1-ee50-4da9-8061-e20f55d53fbb None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:43.119159 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88760b1-ee50-4da9-8061-e20f55d53fbb None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:43.120593 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 690/1366] 10.222.0.13 () {54 vars in 961 bytes} [Mon Jun 27 21:08:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:43.131955 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3ea91-a160-4694-953b-4eb68999e88d None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:43.132234 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3ea91-a160-4694-953b-4eb68999e88d None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:43.132506 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3ea91-a160-4694-953b-4eb68999e88d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:43.162370 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-18e3ea91-a160-4694-953b-4eb68999e88d None None] MFA Rules not processed for user `1366958034d540b4ba2feb421a6afabc`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:43.163600 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-18e3ea91-a160-4694-953b-4eb68999e88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:43.173349 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 677/1367] 10.222.0.13 () {58 vars in 1058 bytes} [Mon Jun 27 21:08:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:08:43.181940 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88760b1-ee50-4da9-8061-e20f55d53fbb None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:08:43.196796 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88760b1-ee50-4da9-8061-e20f55d53fbb None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:08:43.202356 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23237f71-66fe-4d8b-8d67-bc0091f0d75d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:08:43.203116 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23237f71-66fe-4d8b-8d67-bc0091f0d75d service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:43.203400 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23237f71-66fe-4d8b-8d67-bc0091f0d75d service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:43.203676 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23237f71-66fe-4d8b-8d67-bc0091f0d75d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:43.213369 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23237f71-66fe-4d8b-8d67-bc0091f0d75d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:08:43.214910 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23237f71-66fe-4d8b-8d67-bc0091f0d75d service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:08:43.225497 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 691/1368] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:08:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:08:45.324486 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd1b70-c73f-46c6-b9c0-78a928b8feb8 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:45.325560 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd1b70-c73f-46c6-b9c0-78a928b8feb8 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:45.325995 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd1b70-c73f-46c6-b9c0-78a928b8feb8 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:45.327721 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 678/1369] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:08:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:45.333112 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78840ae8-0331-432a-b794-2772f8cb4ed3 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:45.333915 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78840ae8-0331-432a-b794-2772f8cb4ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:45.334422 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78840ae8-0331-432a-b794-2772f8cb4ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:45.363438 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-78840ae8-0331-432a-b794-2772f8cb4ed3 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:45.364809 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-78840ae8-0331-432a-b794-2772f8cb4ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:45.373619 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 692/1370] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:08:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:08:45.419188 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd1b70-c73f-46c6-b9c0-78a928b8feb8 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:08:45.436103 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd1b70-c73f-46c6-b9c0-78a928b8feb8 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:08:45.442064 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb2a55-323f-4eb8-971b-6034cf39f9db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:08:45.442830 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb2a55-323f-4eb8-971b-6034cf39f9db service nova] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:45.443233 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb2a55-323f-4eb8-971b-6034cf39f9db service nova] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:45.443952 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb2a55-323f-4eb8-971b-6034cf39f9db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:45.453495 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb2a55-323f-4eb8-971b-6034cf39f9db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:08:45.455362 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb2a55-323f-4eb8-971b-6034cf39f9db service nova] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:08:45.465954 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 679/1371] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:08:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:08:46.094256 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246529d2-3944-4ee0-8e56-335d4810e347 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:46.094515 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246529d2-3944-4ee0-8e56-335d4810e347 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:46.094799 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246529d2-3944-4ee0-8e56-335d4810e347 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:46.096815 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 693/1372] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:08:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:46.102373 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01472cf-f8f0-41fb-9eac-c58b348dfb6b None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:46.102664 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01472cf-f8f0-41fb-9eac-c58b348dfb6b None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:46.102941 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01472cf-f8f0-41fb-9eac-c58b348dfb6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:46.121975 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-d01472cf-f8f0-41fb-9eac-c58b348dfb6b None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:46.123178 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-d01472cf-f8f0-41fb-9eac-c58b348dfb6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:46.132962 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 680/1373] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:08:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:08:46.492213 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1080e27-502c-4e2b-b8b8-91b23dd103a3 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:46.492961 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1080e27-502c-4e2b-b8b8-91b23dd103a3 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:46.492961 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1080e27-502c-4e2b-b8b8-91b23dd103a3 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:46.494509 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 694/1374] 10.222.0.13 () {54 vars in 963 bytes} [Mon Jun 27 21:08:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:46.499945 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda5e09-576e-4d3b-a409-d71d9421df14 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:46.500522 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda5e09-576e-4d3b-a409-d71d9421df14 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:46.500748 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda5e09-576e-4d3b-a409-d71d9421df14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:46.559455 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-4fda5e09-576e-4d3b-a409-d71d9421df14 None None] MFA Rules not processed for user `340915e7debe4ce6828bd8a477ed6f16`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:46.560828 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-4fda5e09-576e-4d3b-a409-d71d9421df14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:46.624963 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 681/1375] 10.222.0.13 () {58 vars in 1060 bytes} [Mon Jun 27 21:08:46 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:08:46.633067 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1080e27-502c-4e2b-b8b8-91b23dd103a3 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:08:46.644128 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1080e27-502c-4e2b-b8b8-91b23dd103a3 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:08:46.647894 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf70f5d-3863-4c91-b620-04474138be45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:08:46.648696 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf70f5d-3863-4c91-b620-04474138be45 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:46.649223 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf70f5d-3863-4c91-b620-04474138be45 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:46.649611 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf70f5d-3863-4c91-b620-04474138be45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:46.659569 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf70f5d-3863-4c91-b620-04474138be45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:08:46.661381 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf70f5d-3863-4c91-b620-04474138be45 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:08:46.672893 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 695/1376] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:08:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:08:47.603379 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b103b1cb-3def-424b-b89d-aa25e135c022 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:47.603834 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b103b1cb-3def-424b-b89d-aa25e135c022 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:47.604178 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b103b1cb-3def-424b-b89d-aa25e135c022 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:47.605770 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 682/1377] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:47.610956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4af6b1-b09f-41eb-a29c-d10165f136ce None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:47.611578 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4af6b1-b09f-41eb-a29c-d10165f136ce None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:47.611578 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4af6b1-b09f-41eb-a29c-d10165f136ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:47.631385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-de4af6b1-b09f-41eb-a29c-d10165f136ce None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:47.632696 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-de4af6b1-b09f-41eb-a29c-d10165f136ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:47.641129 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 696/1378] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:08:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:08:47.688044 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b103b1cb-3def-424b-b89d-aa25e135c022 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:08:47.703202 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b103b1cb-3def-424b-b89d-aa25e135c022 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:08:47.708527 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26908deb-aac9-4665-ba96-40c61ef6ed74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:08:47.709426 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26908deb-aac9-4665-ba96-40c61ef6ed74 service nova] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:47.709733 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26908deb-aac9-4665-ba96-40c61ef6ed74 service nova] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:47.710012 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26908deb-aac9-4665-ba96-40c61ef6ed74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:47.719076 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26908deb-aac9-4665-ba96-40c61ef6ed74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:08:47.720738 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26908deb-aac9-4665-ba96-40c61ef6ed74 service nova] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:08:47.725191 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:08:47.732447 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 683/1379] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:08:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:08:49.818739 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b869d-2841-4e46-bd4d-3db6a75c4ea2 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:49.819251 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b869d-2841-4e46-bd4d-3db6a75c4ea2 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:49.819251 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b869d-2841-4e46-bd4d-3db6a75c4ea2 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:49.820600 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 697/1380] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:08:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:08:49.826106 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e553e-706a-40ad-af40-53f3e97b5b3e None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:49.826517 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e553e-706a-40ad-af40-53f3e97b5b3e None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:49.826704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e553e-706a-40ad-af40-53f3e97b5b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:49.845303 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-028e553e-706a-40ad-af40-53f3e97b5b3e None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:49.846873 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-028e553e-706a-40ad-af40-53f3e97b5b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:49.855118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 684/1381] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:08:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:08:49.899745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b869d-2841-4e46-bd4d-3db6a75c4ea2 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:08:49.914874 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b869d-2841-4e46-bd4d-3db6a75c4ea2 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:08:49.921422 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0cf3cf-8925-4742-9f4b-1bf4d61ff6be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:08:49.922072 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0cf3cf-8925-4742-9f4b-1bf4d61ff6be service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:49.922351 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0cf3cf-8925-4742-9f4b-1bf4d61ff6be service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:49.922626 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0cf3cf-8925-4742-9f4b-1bf4d61ff6be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:49.931874 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0cf3cf-8925-4742-9f4b-1bf4d61ff6be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:08:49.933509 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0cf3cf-8925-4742-9f4b-1bf4d61ff6be service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:08:49.943254 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 698/1382] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:08:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:08:52.068177 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a734f-f843-4953-9f70-109bacfed2a9 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:52.069236 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a734f-f843-4953-9f70-109bacfed2a9 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:52.069529 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a734f-f843-4953-9f70-109bacfed2a9 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:52.071426 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 685/1383] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:08:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:52.076942 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e3e2f-2237-44a4-89f4-3d5ee3b1caf6 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:52.077448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e3e2f-2237-44a4-89f4-3d5ee3b1caf6 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:52.077542 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e3e2f-2237-44a4-89f4-3d5ee3b1caf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:52.096892 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-f06e3e2f-2237-44a4-89f4-3d5ee3b1caf6 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:52.098219 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-f06e3e2f-2237-44a4-89f4-3d5ee3b1caf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:52.107926 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 699/1384] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:08:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:08:52.149411 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a734f-f843-4953-9f70-109bacfed2a9 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:08:52.166878 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a734f-f843-4953-9f70-109bacfed2a9 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:08:52.173554 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f825b4e0-dd34-4d56-b212-f596854ed0c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:08:52.174415 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825b4e0-dd34-4d56-b212-f596854ed0c4 service nova] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:52.174704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825b4e0-dd34-4d56-b212-f596854ed0c4 service nova] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:52.174979 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825b4e0-dd34-4d56-b212-f596854ed0c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:52.184347 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f825b4e0-dd34-4d56-b212-f596854ed0c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:08:52.185943 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f825b4e0-dd34-4d56-b212-f596854ed0c4 service nova] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:08:52.196828 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 686/1385] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:08:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:08:54.314447 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d471ea2-47cd-42be-bf0a-b008e9a8e3b9 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:54.314986 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d471ea2-47cd-42be-bf0a-b008e9a8e3b9 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:54.314986 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d471ea2-47cd-42be-bf0a-b008e9a8e3b9 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:54.316361 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 700/1386] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:08:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:54.322357 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f081c-132e-4dad-a3ef-41c33efb2848 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:54.323067 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f081c-132e-4dad-a3ef-41c33efb2848 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:54.323067 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f081c-132e-4dad-a3ef-41c33efb2848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:54.342212 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-eb1f081c-132e-4dad-a3ef-41c33efb2848 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:54.343933 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-eb1f081c-132e-4dad-a3ef-41c33efb2848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:54.355358 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 687/1387] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:08:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:08:54.403663 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d471ea2-47cd-42be-bf0a-b008e9a8e3b9 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:08:54.420365 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d471ea2-47cd-42be-bf0a-b008e9a8e3b9 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:08:54.426553 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d93a18-6255-4043-87b4-7a3d9c08a078 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:08:54.427241 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d93a18-6255-4043-87b4-7a3d9c08a078 service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:54.427678 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d93a18-6255-4043-87b4-7a3d9c08a078 service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:54.428624 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d93a18-6255-4043-87b4-7a3d9c08a078 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:54.439211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d93a18-6255-4043-87b4-7a3d9c08a078 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:08:54.441015 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d93a18-6255-4043-87b4-7a3d9c08a078 service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:08:54.458910 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 701/1388] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:08:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:08:56.684615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e5e73-2e48-45a5-badf-0cdd61d8a7bc None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:56.685415 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e5e73-2e48-45a5-badf-0cdd61d8a7bc None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:56.685415 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e5e73-2e48-45a5-badf-0cdd61d8a7bc None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:56.686770 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 688/1389] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:08:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:56.693493 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed12309-e4ed-44b8-8f5a-bae0bc90d9f2 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:56.694161 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed12309-e4ed-44b8-8f5a-bae0bc90d9f2 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:56.694359 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed12309-e4ed-44b8-8f5a-bae0bc90d9f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:56.718002 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-eed12309-e4ed-44b8-8f5a-bae0bc90d9f2 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:56.719672 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-eed12309-e4ed-44b8-8f5a-bae0bc90d9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:56.733106 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 702/1390] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:08:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:08:56.782376 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e5e73-2e48-45a5-badf-0cdd61d8a7bc None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:08:56.799576 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e5e73-2e48-45a5-badf-0cdd61d8a7bc None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:08:56.807880 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce06807-6215-4d46-8d13-fb5929ad956b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:08:56.808590 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce06807-6215-4d46-8d13-fb5929ad956b service nova] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:56.808889 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce06807-6215-4d46-8d13-fb5929ad956b service nova] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:56.809213 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce06807-6215-4d46-8d13-fb5929ad956b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:56.821814 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce06807-6215-4d46-8d13-fb5929ad956b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:08:56.824045 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce06807-6215-4d46-8d13-fb5929ad956b service nova] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:08:56.839278 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 689/1391] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:08:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:08:59.181247 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f18352-eb37-481d-9a56-70682e6f1652 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:59.182052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f18352-eb37-481d-9a56-70682e6f1652 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:59.182052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f18352-eb37-481d-9a56-70682e6f1652 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:59.183379 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 703/1392] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:08:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:08:59.189195 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c0e86-e441-4cf6-b9fd-be135faa9c85 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:59.189666 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c0e86-e441-4cf6-b9fd-be135faa9c85 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:59.189770 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c0e86-e441-4cf6-b9fd-be135faa9c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:59.208964 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-d29c0e86-e441-4cf6-b9fd-be135faa9c85 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:08:59.210290 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-d29c0e86-e441-4cf6-b9fd-be135faa9c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:08:59.220477 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 690/1393] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:08:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:08:59.282159 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f18352-eb37-481d-9a56-70682e6f1652 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:08:59.304194 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f18352-eb37-481d-9a56-70682e6f1652 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:08:59.311656 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944b8063-e816-471e-8430-60c2231c6b38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:08:59.312519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b8063-e816-471e-8430-60c2231c6b38 service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:08:59.312519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b8063-e816-471e-8430-60c2231c6b38 service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:08:59.312797 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b8063-e816-471e-8430-60c2231c6b38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:08:59.324232 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944b8063-e816-471e-8430-60c2231c6b38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:08:59.326023 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944b8063-e816-471e-8430-60c2231c6b38 service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:08:59.338366 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 704/1394] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:08:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:01.546678 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253fb02-fa8a-454d-954c-d11836dc1670 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:01.547228 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253fb02-fa8a-454d-954c-d11836dc1670 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:01.547228 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253fb02-fa8a-454d-954c-d11836dc1670 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:01.548472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 691/1395] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:09:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:01.555124 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97206733-3f32-4faa-8dc6-a4394f489b6e None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:01.555636 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97206733-3f32-4faa-8dc6-a4394f489b6e None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:01.555745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97206733-3f32-4faa-8dc6-a4394f489b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:01.574428 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-97206733-3f32-4faa-8dc6-a4394f489b6e None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:01.575633 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-97206733-3f32-4faa-8dc6-a4394f489b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:01.584586 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 705/1396] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:09:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:01.629642 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253fb02-fa8a-454d-954c-d11836dc1670 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:01.644958 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253fb02-fa8a-454d-954c-d11836dc1670 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:01.651533 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a2167-b856-42fe-8255-a7d559ede110 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:01.652170 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a2167-b856-42fe-8255-a7d559ede110 service nova] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:01.652453 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a2167-b856-42fe-8255-a7d559ede110 service nova] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:01.652725 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a2167-b856-42fe-8255-a7d559ede110 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:01.661272 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a2167-b856-42fe-8255-a7d559ede110 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:01.663417 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a2167-b856-42fe-8255-a7d559ede110 service nova] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:01.673387 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 692/1397] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:01.865154 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786c8bd-3a03-4e13-b558-f7784fd47d5a None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:01.865436 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786c8bd-3a03-4e13-b558-f7784fd47d5a None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:01.865799 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b786c8bd-3a03-4e13-b558-f7784fd47d5a None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:01.867310 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 706/1398] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:09:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:01.873491 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf58b91-9647-4569-a321-7e85cf19ea3e None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:01.873772 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf58b91-9647-4569-a321-7e85cf19ea3e None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:01.874050 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf58b91-9647-4569-a321-7e85cf19ea3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:01.893982 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-edf58b91-9647-4569-a321-7e85cf19ea3e None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:01.895238 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-edf58b91-9647-4569-a321-7e85cf19ea3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:01.904704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 693/1399] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:09:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:01.960567 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052ebb0-8b54-4a65-a5f6-73d53008d6de None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:01.960833 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052ebb0-8b54-4a65-a5f6-73d53008d6de None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:01.961179 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052ebb0-8b54-4a65-a5f6-73d53008d6de None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:01.962571 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 707/1400] 10.222.0.13 () {54 vars in 962 bytes} [Mon Jun 27 21:09:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:01.967893 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335640f4-1e53-480e-83a2-c430007fe66f None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:01.968156 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335640f4-1e53-480e-83a2-c430007fe66f None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:01.968451 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335640f4-1e53-480e-83a2-c430007fe66f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:02.021223 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-335640f4-1e53-480e-83a2-c430007fe66f None None] MFA Rules not processed for user `26cf962d151b4da7a82c33ef23b342e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:02.022446 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-335640f4-1e53-480e-83a2-c430007fe66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:02.087817 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 694/1401] 10.222.0.13 () {58 vars in 1059 bytes} [Mon Jun 27 21:09:01 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:02.096118 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0052ebb0-8b54-4a65-a5f6-73d53008d6de None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:02.108261 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0052ebb0-8b54-4a65-a5f6-73d53008d6de None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:02.111849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e9cbac-44b6-4ec3-9437-4db2c736d91e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26cf962d151b4da7a82c33ef23b342e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:02.112518 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9cbac-44b6-4ec3-9437-4db2c736d91e service glance] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:02.112795 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9cbac-44b6-4ec3-9437-4db2c736d91e service glance] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:02.113097 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9cbac-44b6-4ec3-9437-4db2c736d91e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:02.122917 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e9cbac-44b6-4ec3-9437-4db2c736d91e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:02.124340 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e9cbac-44b6-4ec3-9437-4db2c736d91e service glance] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:02.134703 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 708/1402] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:02.860256 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44452e0-cb69-4d3c-a26a-4bb3c9639633 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:02.860770 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44452e0-cb69-4d3c-a26a-4bb3c9639633 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:02.860882 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44452e0-cb69-4d3c-a26a-4bb3c9639633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:02.911332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-e44452e0-cb69-4d3c-a26a-4bb3c9639633 None None] MFA Rules not processed for user `f67110d591a94537a3df6a2d835370f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:02.912618 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-e44452e0-cb69-4d3c-a26a-4bb3c9639633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:02.958375 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:09:02.984755 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 695/1403] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Jun 27 21:09:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:03.028140 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4e58b-b790-447a-815c-b0f6e411c97a None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:03.028669 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4e58b-b790-447a-815c-b0f6e411c97a None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:03.028669 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4e58b-b790-447a-815c-b0f6e411c97a None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:03.030126 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 709/1404] 10.222.0.13 () {54 vars in 942 bytes} [Mon Jun 27 21:09:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:03.036694 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054712dc-0ee8-4b0c-ba7e-d173a0dfbd32 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:03.037037 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054712dc-0ee8-4b0c-ba7e-d173a0dfbd32 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:03.037379 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054712dc-0ee8-4b0c-ba7e-d173a0dfbd32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:03.089190 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-054712dc-0ee8-4b0c-ba7e-d173a0dfbd32 None None] MFA Rules not processed for user `8bed6b231ee647d8a350da055d8e59d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:03.090426 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-054712dc-0ee8-4b0c-ba7e-d173a0dfbd32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:03.165112 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 696/1405] 10.222.0.13 () {58 vars in 1039 bytes} [Mon Jun 27 21:09:03 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:03.174530 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf4e58b-b790-447a-815c-b0f6e411c97a None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:03.182311 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=61 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:09:03.189080 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf4e58b-b790-447a-815c-b0f6e411c97a None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:03.194325 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc9b6ab-c8fc-408e-89ef-a15ea66e0ed6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bed6b231ee647d8a350da055d8e59d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:03.194956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9b6ab-c8fc-408e-89ef-a15ea66e0ed6 service swift] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:03.195234 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9b6ab-c8fc-408e-89ef-a15ea66e0ed6 service swift] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:03.195505 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc9b6ab-c8fc-408e-89ef-a15ea66e0ed6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:03.204285 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc9b6ab-c8fc-408e-89ef-a15ea66e0ed6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:03.205911 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc9b6ab-c8fc-408e-89ef-a15ea66e0ed6 service swift] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:03.214308 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 710/1406] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Jun 27 21:09:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 21:09:03.787840 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df7ffc-8eb3-4191-b8a5-38f5027283da None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:03.787840 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df7ffc-8eb3-4191-b8a5-38f5027283da None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:03.788326 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df7ffc-8eb3-4191-b8a5-38f5027283da None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:03.794051 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 697/1407] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:09:03 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:09:03.799504 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84783ed9-1bf8-4bca-a5ad-2431bad754f9 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:03.800325 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84783ed9-1bf8-4bca-a5ad-2431bad754f9 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:03.801821 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84783ed9-1bf8-4bca-a5ad-2431bad754f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:03.840029 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-84783ed9-1bf8-4bca-a5ad-2431bad754f9 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:03.841811 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-84783ed9-1bf8-4bca-a5ad-2431bad754f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:03.853657 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 711/1408] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:09:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:03.899086 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0df7ffc-8eb3-4191-b8a5-38f5027283da None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:03.914784 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0df7ffc-8eb3-4191-b8a5-38f5027283da None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:03.923213 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b05e68-6e9c-41b3-836d-b2010c88d9d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:03.923866 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b05e68-6e9c-41b3-836d-b2010c88d9d0 service nova] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:03.924148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b05e68-6e9c-41b3-836d-b2010c88d9d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:03.924419 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b05e68-6e9c-41b3-836d-b2010c88d9d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:03.934251 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b05e68-6e9c-41b3-836d-b2010c88d9d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:03.935843 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b05e68-6e9c-41b3-836d-b2010c88d9d0 service nova] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:03.946948 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 698/1409] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:13.834064 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=8 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:09:13.941462 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=59 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:09:25.187261 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709efc11-8339-4222-9b66-71f78bc92a2a None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:25.187876 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709efc11-8339-4222-9b66-71f78bc92a2a None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:25.187876 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709efc11-8339-4222-9b66-71f78bc92a2a None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:25.189947 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 712/1410] 10.222.0.13 () {54 vars in 918 bytes} [Mon Jun 27 21:09:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:25.195558 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c29e40-6104-4720-b584-f2ee72d8f069 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:25.196076 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c29e40-6104-4720-b584-f2ee72d8f069 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:25.196076 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c29e40-6104-4720-b584-f2ee72d8f069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:25.216296 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-91c29e40-6104-4720-b584-f2ee72d8f069 None None] MFA Rules not processed for user `0437c45d72cd4a38b9d48ee6f50946fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:25.217537 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-91c29e40-6104-4720-b584-f2ee72d8f069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:25.228351 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 699/1411] 10.222.0.13 () {58 vars in 1015 bytes} [Mon Jun 27 21:09:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:25.242729 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9941531-e4c2-461d-b237-8c8ad848c8c9 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:25.243014 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9941531-e4c2-461d-b237-8c8ad848c8c9 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:25.243314 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9941531-e4c2-461d-b237-8c8ad848c8c9 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:25.244651 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 713/1412] 10.222.0.13 () {54 vars in 918 bytes} [Mon Jun 27 21:09:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:25.249376 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a23b2c-3f47-4065-9129-79220dedb885 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:25.249668 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a23b2c-3f47-4065-9129-79220dedb885 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:25.249945 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a23b2c-3f47-4065-9129-79220dedb885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:25.268960 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-78a23b2c-3f47-4065-9129-79220dedb885 None None] MFA Rules not processed for user `1366958034d540b4ba2feb421a6afabc`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:25.270218 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-78a23b2c-3f47-4065-9129-79220dedb885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:25.281326 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 700/1413] 10.222.0.13 () {58 vars in 1015 bytes} [Mon Jun 27 21:09:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:09:27.789788 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77f017-f8cc-451e-b72e-ed5b82d69856 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:27.790370 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77f017-f8cc-451e-b72e-ed5b82d69856 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:27.790370 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77f017-f8cc-451e-b72e-ed5b82d69856 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:27.791707 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 714/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 21:09:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:27.797427 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e1add-0dc0-4280-8e22-b62bd812a9b5 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:27.797899 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e1add-0dc0-4280-8e22-b62bd812a9b5 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:27.797993 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e1add-0dc0-4280-8e22-b62bd812a9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:27.816306 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-b86e1add-0dc0-4280-8e22-b62bd812a9b5 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:27.817559 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-b86e1add-0dc0-4280-8e22-b62bd812a9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:27.826314 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 701/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 21:09:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:09:27.871375 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77f017-f8cc-451e-b72e-ed5b82d69856 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:27.889832 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77f017-f8cc-451e-b72e-ed5b82d69856 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:27.896715 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c13a58-44ad-4f01-b5c9-6926dfaecca0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:27.897442 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c13a58-44ad-4f01-b5c9-6926dfaecca0 service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:27.897704 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c13a58-44ad-4f01-b5c9-6926dfaecca0 service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:27.897995 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c13a58-44ad-4f01-b5c9-6926dfaecca0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:27.918623 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c13a58-44ad-4f01-b5c9-6926dfaecca0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:27.920220 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c13a58-44ad-4f01-b5c9-6926dfaecca0 service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:27.933266 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 715/1416] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:28.868175 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b23aa0-e1a4-4406-9104-b36db79d4a54 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:28.869435 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b23aa0-e1a4-4406-9104-b36db79d4a54 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:28.869650 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b23aa0-e1a4-4406-9104-b36db79d4a54 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:28.870925 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 702/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 21:09:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:28.876291 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36961c-c87b-4420-a702-2e11bccc4d65 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:28.876822 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36961c-c87b-4420-a702-2e11bccc4d65 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:28.876947 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36961c-c87b-4420-a702-2e11bccc4d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:28.896624 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-ee36961c-c87b-4420-a702-2e11bccc4d65 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:28.897873 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ee36961c-c87b-4420-a702-2e11bccc4d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:28.905762 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 716/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 21:09:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:29.055063 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b23aa0-e1a4-4406-9104-b36db79d4a54 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:29.071428 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b23aa0-e1a4-4406-9104-b36db79d4a54 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:29.077293 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d421dd-ce09-48fd-8d6b-cab8b0974f3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:29.078034 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d421dd-ce09-48fd-8d6b-cab8b0974f3e service nova] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:29.078319 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d421dd-ce09-48fd-8d6b-cab8b0974f3e service nova] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:29.078710 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d421dd-ce09-48fd-8d6b-cab8b0974f3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:29.088180 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d421dd-ce09-48fd-8d6b-cab8b0974f3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:29.089835 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d421dd-ce09-48fd-8d6b-cab8b0974f3e service nova] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:29.100290 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 703/1419] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:29.997293 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f740b-1dbd-4c5e-8bf5-219e9dbef7d2 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:29.997896 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f740b-1dbd-4c5e-8bf5-219e9dbef7d2 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:29.997896 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f740b-1dbd-4c5e-8bf5-219e9dbef7d2 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:29.999272 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 717/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 21:09:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:30.004472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea3bed-817c-4857-9248-13c851b562de None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:30.004909 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea3bed-817c-4857-9248-13c851b562de None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:30.005071 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea3bed-817c-4857-9248-13c851b562de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:30.024589 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-b6ea3bed-817c-4857-9248-13c851b562de None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:30.025873 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-b6ea3bed-817c-4857-9248-13c851b562de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:30.034486 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 704/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 21:09:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:30.183432 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f740b-1dbd-4c5e-8bf5-219e9dbef7d2 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:30.198611 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f740b-1dbd-4c5e-8bf5-219e9dbef7d2 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:30.204519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cd37e6-6ee8-4cd0-89a1-c1e3e09e4059 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:30.205212 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd37e6-6ee8-4cd0-89a1-c1e3e09e4059 service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:30.205491 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd37e6-6ee8-4cd0-89a1-c1e3e09e4059 service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:30.205765 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd37e6-6ee8-4cd0-89a1-c1e3e09e4059 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:30.214885 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cd37e6-6ee8-4cd0-89a1-c1e3e09e4059 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:30.216485 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cd37e6-6ee8-4cd0-89a1-c1e3e09e4059 service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:30.227070 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 718/1422] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:31.093517 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b16894-f9c1-4de6-b387-747b9e12cdb3 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:31.094124 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b16894-f9c1-4de6-b387-747b9e12cdb3 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:31.094124 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b16894-f9c1-4de6-b387-747b9e12cdb3 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:31.095828 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 705/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 21:09:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:31.100956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada9e452-dbf9-4116-9931-54682175cdc2 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:31.101447 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada9e452-dbf9-4116-9931-54682175cdc2 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:31.101528 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada9e452-dbf9-4116-9931-54682175cdc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:31.122180 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-ada9e452-dbf9-4116-9931-54682175cdc2 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:31.123412 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ada9e452-dbf9-4116-9931-54682175cdc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:31.132231 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 719/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 21:09:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:31.283243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b16894-f9c1-4de6-b387-747b9e12cdb3 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:31.299691 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b16894-f9c1-4de6-b387-747b9e12cdb3 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:31.305697 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf33de1-f788-4616-9134-c8371587b8cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:31.306349 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf33de1-f788-4616-9134-c8371587b8cd service nova] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:31.306629 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf33de1-f788-4616-9134-c8371587b8cd service nova] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:31.306905 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf33de1-f788-4616-9134-c8371587b8cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:31.315965 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf33de1-f788-4616-9134-c8371587b8cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:31.317994 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf33de1-f788-4616-9134-c8371587b8cd service nova] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:31.328521 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 706/1425] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:32.202700 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10bab4-3051-4e7d-b5d0-206f3e1349a7 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:32.203248 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10bab4-3051-4e7d-b5d0-206f3e1349a7 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:32.203248 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10bab4-3051-4e7d-b5d0-206f3e1349a7 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:32.204742 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 720/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 21:09:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:32.209856 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a5bee-3776-4d14-aa72-12fa42ae0b83 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:32.210314 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a5bee-3776-4d14-aa72-12fa42ae0b83 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:32.210410 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a5bee-3776-4d14-aa72-12fa42ae0b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:32.230391 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-877a5bee-3776-4d14-aa72-12fa42ae0b83 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:32.231606 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-877a5bee-3776-4d14-aa72-12fa42ae0b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:32.240902 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 707/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 21:09:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:32.392945 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d10bab4-3051-4e7d-b5d0-206f3e1349a7 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:32.409963 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d10bab4-3051-4e7d-b5d0-206f3e1349a7 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:32.415766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff2b8d2-232a-4fc3-a470-02a74296414d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:32.416412 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2b8d2-232a-4fc3-a470-02a74296414d service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:32.416691 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2b8d2-232a-4fc3-a470-02a74296414d service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:32.416964 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2b8d2-232a-4fc3-a470-02a74296414d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:32.426049 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff2b8d2-232a-4fc3-a470-02a74296414d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:32.427899 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff2b8d2-232a-4fc3-a470-02a74296414d service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:32.438412 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 721/1428] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:33.249788 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff61cd-6697-42df-99bb-46e4e741302b None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:33.250390 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff61cd-6697-42df-99bb-46e4e741302b None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:33.250390 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff61cd-6697-42df-99bb-46e4e741302b None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:33.251724 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 708/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 21:09:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:33.257085 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9612717-5188-4c97-b4db-0558bbb7ad99 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:33.257539 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9612717-5188-4c97-b4db-0558bbb7ad99 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:33.257638 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9612717-5188-4c97-b4db-0558bbb7ad99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:33.277408 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-b9612717-5188-4c97-b4db-0558bbb7ad99 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:33.278549 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-b9612717-5188-4c97-b4db-0558bbb7ad99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:33.287611 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 722/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 21:09:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:33.436652 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff61cd-6697-42df-99bb-46e4e741302b None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:33.451676 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff61cd-6697-42df-99bb-46e4e741302b None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:33.457122 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca9219a-a2b6-45e1-9ce4-fdfe207f14a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:33.457769 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca9219a-a2b6-45e1-9ce4-fdfe207f14a3 service nova] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:33.458053 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca9219a-a2b6-45e1-9ce4-fdfe207f14a3 service nova] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:33.458353 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca9219a-a2b6-45e1-9ce4-fdfe207f14a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:33.466631 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca9219a-a2b6-45e1-9ce4-fdfe207f14a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:33.468218 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca9219a-a2b6-45e1-9ce4-fdfe207f14a3 service nova] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:33.478485 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 709/1431] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:34.279971 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094e8b4-71ed-4702-96a0-afd10039b129 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:34.280587 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094e8b4-71ed-4702-96a0-afd10039b129 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:34.280587 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094e8b4-71ed-4702-96a0-afd10039b129 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:34.282043 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 723/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 21:09:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:34.287501 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601cdac9-a54b-4371-beb2-72983389a855 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:34.287918 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601cdac9-a54b-4371-beb2-72983389a855 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:34.288088 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601cdac9-a54b-4371-beb2-72983389a855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:34.307541 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-601cdac9-a54b-4371-beb2-72983389a855 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:34.308821 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-601cdac9-a54b-4371-beb2-72983389a855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:34.318277 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 710/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 21:09:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:34.467714 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094e8b4-71ed-4702-96a0-afd10039b129 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:34.483310 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094e8b4-71ed-4702-96a0-afd10039b129 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:34.488981 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d353a2d-9a86-480d-930e-e1a657f4f0f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:34.489923 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d353a2d-9a86-480d-930e-e1a657f4f0f2 service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:34.490307 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d353a2d-9a86-480d-930e-e1a657f4f0f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:34.490587 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d353a2d-9a86-480d-930e-e1a657f4f0f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:34.500268 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d353a2d-9a86-480d-930e-e1a657f4f0f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:34.501916 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d353a2d-9a86-480d-930e-e1a657f4f0f2 service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:34.512287 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 724/1434] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:35.329950 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96758cdb-98f5-4ad9-b2d6-73601a74280c None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:35.330428 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96758cdb-98f5-4ad9-b2d6-73601a74280c None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:35.330560 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96758cdb-98f5-4ad9-b2d6-73601a74280c None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:35.331880 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 711/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 21:09:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:35.336949 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed25c9-e4de-4b1f-825e-b379e16717f7 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:35.337386 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed25c9-e4de-4b1f-825e-b379e16717f7 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:35.337480 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed25c9-e4de-4b1f-825e-b379e16717f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:35.356909 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-50ed25c9-e4de-4b1f-825e-b379e16717f7 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:35.358144 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-50ed25c9-e4de-4b1f-825e-b379e16717f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:35.367388 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 725/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 21:09:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:35.519325 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96758cdb-98f5-4ad9-b2d6-73601a74280c None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:35.536220 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96758cdb-98f5-4ad9-b2d6-73601a74280c None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:35.542707 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e4e23-2734-4a21-8882-8dc440cf3709 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:35.543356 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e4e23-2734-4a21-8882-8dc440cf3709 service nova] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:35.543613 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e4e23-2734-4a21-8882-8dc440cf3709 service nova] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:35.543916 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e4e23-2734-4a21-8882-8dc440cf3709 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:35.554184 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e4e23-2734-4a21-8882-8dc440cf3709 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:35.556223 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e4e23-2734-4a21-8882-8dc440cf3709 service nova] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:35.568206 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 712/1437] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:36.394987 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4c58f-5b2e-4e2e-bf75-4a1846e1d67e None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:36.395553 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4c58f-5b2e-4e2e-bf75-4a1846e1d67e None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:36.395553 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4c58f-5b2e-4e2e-bf75-4a1846e1d67e None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:36.397145 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 726/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 21:09:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:36.402437 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87650cb-4270-44ca-b739-9207291fc014 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:36.402915 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87650cb-4270-44ca-b739-9207291fc014 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:36.403033 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87650cb-4270-44ca-b739-9207291fc014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:36.422003 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-d87650cb-4270-44ca-b739-9207291fc014 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:36.423230 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-d87650cb-4270-44ca-b739-9207291fc014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:36.432589 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 713/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 21:09:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:36.596356 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c4c58f-5b2e-4e2e-bf75-4a1846e1d67e None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:36.615159 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c4c58f-5b2e-4e2e-bf75-4a1846e1d67e None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:36.620744 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fc7063-be2d-47fb-b4b9-0a9e5a8cfc63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:36.621438 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc7063-be2d-47fb-b4b9-0a9e5a8cfc63 service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:36.621720 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc7063-be2d-47fb-b4b9-0a9e5a8cfc63 service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:36.621991 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc7063-be2d-47fb-b4b9-0a9e5a8cfc63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:36.632229 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fc7063-be2d-47fb-b4b9-0a9e5a8cfc63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:36.633891 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fc7063-be2d-47fb-b4b9-0a9e5a8cfc63 service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:36.644680 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 727/1440] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:38.619623 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91481b1-4858-45a7-944e-290215809681 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:38.620219 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91481b1-4858-45a7-944e-290215809681 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:38.620219 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91481b1-4858-45a7-944e-290215809681 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:38.621543 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 714/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 21:09:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:38.626926 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13ca5d-20ec-4afb-8236-2a69f3247570 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:38.627380 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13ca5d-20ec-4afb-8236-2a69f3247570 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:38.627487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13ca5d-20ec-4afb-8236-2a69f3247570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:38.645835 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-1a13ca5d-20ec-4afb-8236-2a69f3247570 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:38.647051 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-1a13ca5d-20ec-4afb-8236-2a69f3247570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:38.654992 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 728/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 21:09:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:38.694425 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91481b1-4858-45a7-944e-290215809681 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:38.705865 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91481b1-4858-45a7-944e-290215809681 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:38.709972 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d014049-47c5-41eb-9f30-81e735735563 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:38.710617 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d014049-47c5-41eb-9f30-81e735735563 service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:38.710901 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d014049-47c5-41eb-9f30-81e735735563 service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:38.711174 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d014049-47c5-41eb-9f30-81e735735563 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:38.721550 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d014049-47c5-41eb-9f30-81e735735563 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:38.722987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d014049-47c5-41eb-9f30-81e735735563 service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:38.733604 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 715/1443] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:41.103599 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a590f-47a8-4935-8767-cda740e1f09e None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:41.104211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a590f-47a8-4935-8767-cda740e1f09e None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:41.104211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a590f-47a8-4935-8767-cda740e1f09e None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:41.105574 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 729/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 21:09:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:41.111204 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e84a4-cd34-4ba6-8b17-4d9172c1047c None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:41.111729 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e84a4-cd34-4ba6-8b17-4d9172c1047c None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:41.111729 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e84a4-cd34-4ba6-8b17-4d9172c1047c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:41.130293 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-c31e84a4-cd34-4ba6-8b17-4d9172c1047c None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:41.131494 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-c31e84a4-cd34-4ba6-8b17-4d9172c1047c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:41.139808 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 716/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 21:09:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:41.179562 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a590f-47a8-4935-8767-cda740e1f09e None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:41.191647 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a590f-47a8-4935-8767-cda740e1f09e None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:41.195792 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d96be-aa3a-4902-9cab-c8d93cb08a41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:41.196449 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d96be-aa3a-4902-9cab-c8d93cb08a41 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:41.196733 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d96be-aa3a-4902-9cab-c8d93cb08a41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:41.197042 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d96be-aa3a-4902-9cab-c8d93cb08a41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:41.207092 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237d96be-aa3a-4902-9cab-c8d93cb08a41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:41.208518 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237d96be-aa3a-4902-9cab-c8d93cb08a41 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:41.219169 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 730/1446] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:43.606273 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c80a13-5ec7-48de-b4a8-e6d747847a67 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:43.606831 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c80a13-5ec7-48de-b4a8-e6d747847a67 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:43.606831 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c80a13-5ec7-48de-b4a8-e6d747847a67 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:43.608081 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 717/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 21:09:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:43.613794 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb341a-24f9-4e86-abe6-5b09cb674482 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:43.614310 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb341a-24f9-4e86-abe6-5b09cb674482 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:43.614726 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb341a-24f9-4e86-abe6-5b09cb674482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:43.634695 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-86eb341a-24f9-4e86-abe6-5b09cb674482 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:43.635946 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-86eb341a-24f9-4e86-abe6-5b09cb674482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:43.645300 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 731/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 21:09:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:43.684300 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c80a13-5ec7-48de-b4a8-e6d747847a67 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:43.696829 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c80a13-5ec7-48de-b4a8-e6d747847a67 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:43.701529 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69fd212-66af-48db-b4dd-032072bc86d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:43.702187 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69fd212-66af-48db-b4dd-032072bc86d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:43.702468 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69fd212-66af-48db-b4dd-032072bc86d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:43.702740 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69fd212-66af-48db-b4dd-032072bc86d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:43.713443 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69fd212-66af-48db-b4dd-032072bc86d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:43.714901 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69fd212-66af-48db-b4dd-032072bc86d5 service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:43.726049 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 718/1449] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:45.902630 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf24d76-37c0-460b-8905-adfcc86dee11 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:45.903283 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf24d76-37c0-460b-8905-adfcc86dee11 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:45.903283 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf24d76-37c0-460b-8905-adfcc86dee11 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:45.904617 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 732/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 21:09:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:45.910946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d0b895-7495-41d3-b6a8-ad11000d1839 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:45.911454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d0b895-7495-41d3-b6a8-ad11000d1839 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:45.911730 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d0b895-7495-41d3-b6a8-ad11000d1839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:45.933352 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-f6d0b895-7495-41d3-b6a8-ad11000d1839 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:45.934585 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-f6d0b895-7495-41d3-b6a8-ad11000d1839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:45.943606 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 719/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 21:09:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:45.985567 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf24d76-37c0-460b-8905-adfcc86dee11 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:45.998414 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf24d76-37c0-460b-8905-adfcc86dee11 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:46.002481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962d240e-806c-48b2-940b-3978670b1ff2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:46.003221 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d240e-806c-48b2-940b-3978670b1ff2 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:46.003602 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d240e-806c-48b2-940b-3978670b1ff2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:46.003972 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d240e-806c-48b2-940b-3978670b1ff2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:46.014733 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962d240e-806c-48b2-940b-3978670b1ff2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:46.016234 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962d240e-806c-48b2-940b-3978670b1ff2 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:46.027523 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 733/1452] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:48.149344 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc752a-f702-4ce4-86fd-0e7e7be271c9 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:48.149964 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc752a-f702-4ce4-86fd-0e7e7be271c9 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:48.149964 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc752a-f702-4ce4-86fd-0e7e7be271c9 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:48.151282 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 720/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 21:09:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:48.156722 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb50b07-04cf-4299-864f-27aef78a3294 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:48.157193 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb50b07-04cf-4299-864f-27aef78a3294 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:48.157314 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb50b07-04cf-4299-864f-27aef78a3294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:48.175683 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-bbb50b07-04cf-4299-864f-27aef78a3294 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:48.177017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-bbb50b07-04cf-4299-864f-27aef78a3294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:48.185518 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 734/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 21:09:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:48.224047 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bc752a-f702-4ce4-86fd-0e7e7be271c9 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:48.236395 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bc752a-f702-4ce4-86fd-0e7e7be271c9 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:48.240860 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc00145-6107-47bc-86d6-771cebd2813f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:48.241544 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc00145-6107-47bc-86d6-771cebd2813f service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:48.241827 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc00145-6107-47bc-86d6-771cebd2813f service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:48.242099 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc00145-6107-47bc-86d6-771cebd2813f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:48.252501 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc00145-6107-47bc-86d6-771cebd2813f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:48.253965 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc00145-6107-47bc-86d6-771cebd2813f service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:48.263984 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 721/1455] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:50.384292 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b827a-be1b-4443-9f28-a571f4f108fc None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:50.384836 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b827a-be1b-4443-9f28-a571f4f108fc None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:50.384836 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b827a-be1b-4443-9f28-a571f4f108fc None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:50.386143 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 735/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 21:09:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:50.391507 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57602a8-2108-441c-a40c-da530faf54f5 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:50.391969 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57602a8-2108-441c-a40c-da530faf54f5 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:50.392088 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57602a8-2108-441c-a40c-da530faf54f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:50.412499 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-d57602a8-2108-441c-a40c-da530faf54f5 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:50.413743 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-d57602a8-2108-441c-a40c-da530faf54f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:50.422846 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 722/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 21:09:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:50.461599 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b827a-be1b-4443-9f28-a571f4f108fc None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:50.473824 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b827a-be1b-4443-9f28-a571f4f108fc None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:50.478331 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce24f19-d7f6-441d-bdba-47424e59b1f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:50.478975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce24f19-d7f6-441d-bdba-47424e59b1f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:50.479234 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce24f19-d7f6-441d-bdba-47424e59b1f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:50.479511 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce24f19-d7f6-441d-bdba-47424e59b1f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:50.489911 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce24f19-d7f6-441d-bdba-47424e59b1f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:50.491401 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce24f19-d7f6-441d-bdba-47424e59b1f6 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:50.502115 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 736/1458] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:52.625908 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a723a60-b7f6-41d5-a4e4-25eee724298b None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:52.626510 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a723a60-b7f6-41d5-a4e4-25eee724298b None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:52.626510 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a723a60-b7f6-41d5-a4e4-25eee724298b None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:52.627847 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 723/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 21:09:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:52.633507 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f99b445-500b-418b-a642-883ce445dda7 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:52.633938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f99b445-500b-418b-a642-883ce445dda7 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:52.634056 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f99b445-500b-418b-a642-883ce445dda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:52.653260 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-1f99b445-500b-418b-a642-883ce445dda7 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:52.654462 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-1f99b445-500b-418b-a642-883ce445dda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:52.663364 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 737/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 21:09:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:52.704445 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a723a60-b7f6-41d5-a4e4-25eee724298b None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:52.717126 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a723a60-b7f6-41d5-a4e4-25eee724298b None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:52.721287 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c00c3f-a41b-4122-8c93-e806e465b0d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:52.721939 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c00c3f-a41b-4122-8c93-e806e465b0d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:52.722221 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c00c3f-a41b-4122-8c93-e806e465b0d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:52.722494 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c00c3f-a41b-4122-8c93-e806e465b0d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:52.736351 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c00c3f-a41b-4122-8c93-e806e465b0d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:52.738540 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c00c3f-a41b-4122-8c93-e806e465b0d5 service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:52.749186 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 724/1461] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:54.880587 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a481cb0-3062-47bf-b6a6-5b906c5e1c46 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:54.881192 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a481cb0-3062-47bf-b6a6-5b906c5e1c46 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:54.881192 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a481cb0-3062-47bf-b6a6-5b906c5e1c46 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:54.882573 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 738/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 21:09:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:54.888305 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e93c1e-daa7-4799-9a9c-ca9057aa0079 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:54.888721 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e93c1e-daa7-4799-9a9c-ca9057aa0079 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:54.889008 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e93c1e-daa7-4799-9a9c-ca9057aa0079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:54.895989 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=79 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:09:54.908839 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-74e93c1e-daa7-4799-9a9c-ca9057aa0079 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:54.910098 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-74e93c1e-daa7-4799-9a9c-ca9057aa0079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:54.919572 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 725/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 21:09:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:54.957928 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a481cb0-3062-47bf-b6a6-5b906c5e1c46 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:54.964130 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=73 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:09:54.971398 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a481cb0-3062-47bf-b6a6-5b906c5e1c46 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:54.975131 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ce5197-5c03-4890-b71a-b207ef98d117 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:54.975773 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ce5197-5c03-4890-b71a-b207ef98d117 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:54.976051 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ce5197-5c03-4890-b71a-b207ef98d117 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:54.976322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ce5197-5c03-4890-b71a-b207ef98d117 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:54.985850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ce5197-5c03-4890-b71a-b207ef98d117 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:54.987291 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ce5197-5c03-4890-b71a-b207ef98d117 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:54.997353 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 739/1464] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:57.074073 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a07f7-d4a9-4a1a-bbbc-2f0ef85b7c47 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:57.074640 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a07f7-d4a9-4a1a-bbbc-2f0ef85b7c47 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:57.074640 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a07f7-d4a9-4a1a-bbbc-2f0ef85b7c47 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:57.076053 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 726/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 21:09:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:57.081775 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61496e-f2de-4f53-8073-b22f251f0824 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:57.082205 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61496e-f2de-4f53-8073-b22f251f0824 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:57.082314 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61496e-f2de-4f53-8073-b22f251f0824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:57.102260 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-4d61496e-f2de-4f53-8073-b22f251f0824 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:57.103462 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-4d61496e-f2de-4f53-8073-b22f251f0824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:57.112269 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 740/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 21:09:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:57.149996 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a07f7-d4a9-4a1a-bbbc-2f0ef85b7c47 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:57.162308 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a07f7-d4a9-4a1a-bbbc-2f0ef85b7c47 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:57.166615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0042dc03-0ba5-403d-9a4d-b341b2a93b4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:57.167260 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042dc03-0ba5-403d-9a4d-b341b2a93b4f service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:57.167544 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042dc03-0ba5-403d-9a4d-b341b2a93b4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:57.167820 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042dc03-0ba5-403d-9a4d-b341b2a93b4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:57.178432 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0042dc03-0ba5-403d-9a4d-b341b2a93b4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:57.179861 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0042dc03-0ba5-403d-9a4d-b341b2a93b4f service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:57.189476 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 727/1467] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:09:59.240500 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee3561-6c8a-452a-8d3f-67cd99689afa None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:59.241139 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee3561-6c8a-452a-8d3f-67cd99689afa None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:59.241139 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee3561-6c8a-452a-8d3f-67cd99689afa None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:59.242474 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 741/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 21:09:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:09:59.248133 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309987c3-9bff-4347-8ea6-d8acaab76de0 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:59.249218 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309987c3-9bff-4347-8ea6-d8acaab76de0 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:59.249218 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309987c3-9bff-4347-8ea6-d8acaab76de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:59.267529 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-309987c3-9bff-4347-8ea6-d8acaab76de0 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:09:59.268808 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-309987c3-9bff-4347-8ea6-d8acaab76de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:09:59.277264 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 728/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 21:09:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:09:59.313915 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee3561-6c8a-452a-8d3f-67cd99689afa None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:09:59.326006 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee3561-6c8a-452a-8d3f-67cd99689afa None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:09:59.329918 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae7815-0497-42bd-9fc0-cf1ca0240fd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:09:59.330566 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae7815-0497-42bd-9fc0-cf1ca0240fd6 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:09:59.330840 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae7815-0497-42bd-9fc0-cf1ca0240fd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:09:59.331120 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae7815-0497-42bd-9fc0-cf1ca0240fd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:09:59.341185 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eae7815-0497-42bd-9fc0-cf1ca0240fd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:09:59.342624 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eae7815-0497-42bd-9fc0-cf1ca0240fd6 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:09:59.352644 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 742/1470] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:09:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:10:09.262300 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=11 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:10:09.348447 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=14 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:10:19.279767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327da4cd-dd36-444f-a9a9-ffead2941e13 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:19.280361 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327da4cd-dd36-444f-a9a9-ffead2941e13 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:19.280361 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327da4cd-dd36-444f-a9a9-ffead2941e13 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:19.281613 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 729/1471] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:10:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:10:19.288053 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec1b99-f434-480b-86bb-78cfc8582f05 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:19.288542 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec1b99-f434-480b-86bb-78cfc8582f05 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:19.288644 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec1b99-f434-480b-86bb-78cfc8582f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:19.308208 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-b6ec1b99-f434-480b-86bb-78cfc8582f05 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:10:19.309446 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-b6ec1b99-f434-480b-86bb-78cfc8582f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:10:19.318715 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 743/1472] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:10:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:10:19.368064 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0098b3-42ee-48a8-aad1-370ab607d00b None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:19.368356 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0098b3-42ee-48a8-aad1-370ab607d00b None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:19.368631 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0098b3-42ee-48a8-aad1-370ab607d00b None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:19.370035 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 730/1473] 10.222.0.13 () {54 vars in 962 bytes} [Mon Jun 27 21:10:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:10:19.375388 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3aaec1-7048-42c3-a20d-d8dbece3e190 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:19.375708 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3aaec1-7048-42c3-a20d-d8dbece3e190 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:19.375986 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3aaec1-7048-42c3-a20d-d8dbece3e190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:19.394927 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-5a3aaec1-7048-42c3-a20d-d8dbece3e190 None None] MFA Rules not processed for user `26cf962d151b4da7a82c33ef23b342e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:10:19.396142 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-5a3aaec1-7048-42c3-a20d-d8dbece3e190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:10:19.404040 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 744/1474] 10.222.0.13 () {58 vars in 1059 bytes} [Mon Jun 27 21:10:19 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:10:19.412871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0098b3-42ee-48a8-aad1-370ab607d00b None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:10:19.424842 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0098b3-42ee-48a8-aad1-370ab607d00b None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:10:19.428889 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d1a6df-c048-4683-9a2e-40a5f25a0299 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26cf962d151b4da7a82c33ef23b342e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:10:19.429569 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1a6df-c048-4683-9a2e-40a5f25a0299 service glance] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:19.429853 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1a6df-c048-4683-9a2e-40a5f25a0299 service glance] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:19.430128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1a6df-c048-4683-9a2e-40a5f25a0299 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:19.439748 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d1a6df-c048-4683-9a2e-40a5f25a0299 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:10:19.441220 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d1a6df-c048-4683-9a2e-40a5f25a0299 service glance] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:10:19.451103 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 731/1475] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:10:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:10:21.595642 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6fd1f5-6fd9-4e2b-8188-84881fd23f04 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:21.596180 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6fd1f5-6fd9-4e2b-8188-84881fd23f04 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:21.596180 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6fd1f5-6fd9-4e2b-8188-84881fd23f04 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:21.597616 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 745/1476] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:10:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:10:21.603449 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb1d52-91e9-45e8-a210-81cfaa8b5c11 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:21.603893 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb1d52-91e9-45e8-a210-81cfaa8b5c11 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:21.603999 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb1d52-91e9-45e8-a210-81cfaa8b5c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:21.623228 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-3afb1d52-91e9-45e8-a210-81cfaa8b5c11 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:10:21.624450 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-3afb1d52-91e9-45e8-a210-81cfaa8b5c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:10:21.633945 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 732/1477] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:10:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:10:21.657957 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6fd1f5-6fd9-4e2b-8188-84881fd23f04 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:10:21.671595 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6fd1f5-6fd9-4e2b-8188-84881fd23f04 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:10:21.677369 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d98095-6f3b-4077-85e8-d659d8de6052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:10:21.678035 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d98095-6f3b-4077-85e8-d659d8de6052 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:21.678323 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d98095-6f3b-4077-85e8-d659d8de6052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:21.678582 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d98095-6f3b-4077-85e8-d659d8de6052 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:21.679322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d98095-6f3b-4077-85e8-d659d8de6052 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:10:21.681185 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: WARNING py.warnings [None req-91d98095-6f3b-4077-85e8-d659d8de6052 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 21:10:21.681185 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: warnings.warn(msg) Jun 27 21:10:21.681185 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: Jun 27 21:10:21.682329 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d98095-6f3b-4077-85e8-d659d8de6052 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:10:21.703120 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 746/1478] 10.222.0.13 () {56 vars in 1108 bytes} [Mon Jun 27 21:10:21 2022] GET /identity/v3/projects => generated 2553 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:10:23.772244 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505b184-386b-42c6-9c94-296cce5efe89 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:23.772870 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505b184-386b-42c6-9c94-296cce5efe89 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:23.772870 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505b184-386b-42c6-9c94-296cce5efe89 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:23.774209 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 733/1479] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:10:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:10:23.779886 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a6063-2056-4143-a306-641c2eaaa78a None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:23.780345 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a6063-2056-4143-a306-641c2eaaa78a None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:23.780449 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a6063-2056-4143-a306-641c2eaaa78a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:23.800089 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-e02a6063-2056-4143-a306-641c2eaaa78a None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:10:23.801489 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-e02a6063-2056-4143-a306-641c2eaaa78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:10:23.810882 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 747/1480] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:10:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:10:23.854837 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a505b184-386b-42c6-9c94-296cce5efe89 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:10:23.871459 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a505b184-386b-42c6-9c94-296cce5efe89 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:10:23.877680 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf172d-0fbe-47df-8b4a-4b5932fd47d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:10:23.878330 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf172d-0fbe-47df-8b4a-4b5932fd47d0 service nova] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:23.878612 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf172d-0fbe-47df-8b4a-4b5932fd47d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:23.878892 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf172d-0fbe-47df-8b4a-4b5932fd47d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:23.887709 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf172d-0fbe-47df-8b4a-4b5932fd47d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:10:23.889335 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf172d-0fbe-47df-8b4a-4b5932fd47d0 service nova] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:10:23.899422 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 734/1481] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:10:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:10:26.111391 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257288d5-2b23-495c-b9a9-700f19e810c6 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:26.111966 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257288d5-2b23-495c-b9a9-700f19e810c6 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:26.112977 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257288d5-2b23-495c-b9a9-700f19e810c6 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:26.115147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 748/1482] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:10:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:10:26.121356 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd58b1-c5dd-434e-8bbc-b1494461d829 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:26.121898 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd58b1-c5dd-434e-8bbc-b1494461d829 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:26.121898 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd58b1-c5dd-434e-8bbc-b1494461d829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:26.143025 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-87bd58b1-c5dd-434e-8bbc-b1494461d829 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:10:26.144247 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-87bd58b1-c5dd-434e-8bbc-b1494461d829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:10:26.158043 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 735/1483] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:10:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:10:26.194703 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257288d5-2b23-495c-b9a9-700f19e810c6 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:10:26.207414 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257288d5-2b23-495c-b9a9-700f19e810c6 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:10:26.212264 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d84eb4-1ecf-4f98-a25f-1150c186d694 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:10:26.213007 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d84eb4-1ecf-4f98-a25f-1150c186d694 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:26.213365 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d84eb4-1ecf-4f98-a25f-1150c186d694 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:26.213654 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d84eb4-1ecf-4f98-a25f-1150c186d694 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:26.226199 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d84eb4-1ecf-4f98-a25f-1150c186d694 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:10:26.227648 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d84eb4-1ecf-4f98-a25f-1150c186d694 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:10:26.238084 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 749/1484] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:10:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:10:28.361078 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35abd2f-b0a1-4a1f-9697-ec83e9221594 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:28.361887 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35abd2f-b0a1-4a1f-9697-ec83e9221594 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:28.361887 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35abd2f-b0a1-4a1f-9697-ec83e9221594 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:28.362897 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 736/1485] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:10:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:10:28.368614 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1cfb3d-ccd9-47b7-b424-b33656091ca6 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:28.369200 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1cfb3d-ccd9-47b7-b424-b33656091ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:28.369200 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1cfb3d-ccd9-47b7-b424-b33656091ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:28.389800 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-4a1cfb3d-ccd9-47b7-b424-b33656091ca6 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:10:28.391040 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-4a1cfb3d-ccd9-47b7-b424-b33656091ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:10:28.400157 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 750/1486] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:10:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:10:28.438288 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35abd2f-b0a1-4a1f-9697-ec83e9221594 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:10:28.449783 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35abd2f-b0a1-4a1f-9697-ec83e9221594 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:10:28.453652 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906cae91-306c-4dab-a876-a5b445381ee7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:10:28.454303 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906cae91-306c-4dab-a876-a5b445381ee7 service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:28.454585 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906cae91-306c-4dab-a876-a5b445381ee7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:28.454858 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906cae91-306c-4dab-a876-a5b445381ee7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:28.464975 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906cae91-306c-4dab-a876-a5b445381ee7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:10:28.466417 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906cae91-306c-4dab-a876-a5b445381ee7 service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:10:28.477824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 737/1487] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:10:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:10:30.754412 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408d406-1363-44a6-b905-789da10d1242 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:30.755209 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408d406-1363-44a6-b905-789da10d1242 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:30.755209 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408d406-1363-44a6-b905-789da10d1242 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:30.756494 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 751/1488] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:10:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:10:30.762641 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b63f0-11cf-479e-ac9e-db92f87029c2 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:30.763190 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b63f0-11cf-479e-ac9e-db92f87029c2 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:30.763315 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b63f0-11cf-479e-ac9e-db92f87029c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:30.782011 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-dc1b63f0-11cf-479e-ac9e-db92f87029c2 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:10:30.783237 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-dc1b63f0-11cf-479e-ac9e-db92f87029c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:10:30.792125 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 738/1489] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:10:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:10:30.832400 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f408d406-1363-44a6-b905-789da10d1242 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:10:30.846181 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f408d406-1363-44a6-b905-789da10d1242 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:10:30.850879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f15b4c-7af7-4b1d-8f00-7963fab94989 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:10:30.851523 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f15b4c-7af7-4b1d-8f00-7963fab94989 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:30.851804 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f15b4c-7af7-4b1d-8f00-7963fab94989 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:30.852076 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f15b4c-7af7-4b1d-8f00-7963fab94989 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:30.861914 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f15b4c-7af7-4b1d-8f00-7963fab94989 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:10:30.863366 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f15b4c-7af7-4b1d-8f00-7963fab94989 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:10:30.874413 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 752/1490] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:10:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:10:33.250300 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841779eb-5746-4308-b62e-91c4833e59ba None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:33.250874 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841779eb-5746-4308-b62e-91c4833e59ba None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:33.250874 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841779eb-5746-4308-b62e-91c4833e59ba None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:33.252247 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 739/1491] 10.222.0.13 () {54 vars in 926 bytes} [Mon Jun 27 21:10:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:10:33.258666 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794b2bb-f60c-4bc2-ac6b-acb3a7177bee None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:33.259209 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794b2bb-f60c-4bc2-ac6b-acb3a7177bee None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:33.259209 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794b2bb-f60c-4bc2-ac6b-acb3a7177bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:33.279232 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-c794b2bb-f60c-4bc2-ac6b-acb3a7177bee None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:10:33.280362 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-c794b2bb-f60c-4bc2-ac6b-acb3a7177bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:10:33.289793 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 753/1492] 10.222.0.13 () {58 vars in 1023 bytes} [Mon Jun 27 21:10:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:10:33.329150 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841779eb-5746-4308-b62e-91c4833e59ba None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:10:33.341822 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841779eb-5746-4308-b62e-91c4833e59ba None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:10:33.346128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c3da82-5e5a-401f-b9b6-f3c6a61c4325 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:10:33.346784 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3da82-5e5a-401f-b9b6-f3c6a61c4325 service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:10:33.347071 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3da82-5e5a-401f-b9b6-f3c6a61c4325 service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:10:33.347348 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3da82-5e5a-401f-b9b6-f3c6a61c4325 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:10:33.358237 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c3da82-5e5a-401f-b9b6-f3c6a61c4325 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:10:33.359669 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c3da82-5e5a-401f-b9b6-f3c6a61c4325 service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:10:33.370114 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 740/1493] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:10:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:10:43.272748 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=33 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:10:43.365613 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=33 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:12:03.862781 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4364a-d584-47f5-96ae-924767092411 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:12:03.863415 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4364a-d584-47f5-96ae-924767092411 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:12:03.863415 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4364a-d584-47f5-96ae-924767092411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:12:03.882116 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-a5d4364a-d584-47f5-96ae-924767092411 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:12:03.883647 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a5d4364a-d584-47f5-96ae-924767092411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:12:03.892275 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 754/1494] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:12:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:12:03.900625 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d88774-0afd-42a0-a2de-568f609396f8 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:12:03.901200 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d88774-0afd-42a0-a2de-568f609396f8 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:12:03.901200 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d88774-0afd-42a0-a2de-568f609396f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:12:03.920789 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-73d88774-0afd-42a0-a2de-568f609396f8 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:12:03.922035 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-73d88774-0afd-42a0-a2de-568f609396f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:12:03.931671 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 741/1495] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:12:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:12:03.936397 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d4364a-d584-47f5-96ae-924767092411 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:12:03.946811 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d4364a-d584-47f5-96ae-924767092411 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:12:03.952636 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ace9f-33e4-46bb-ab60-f4fd0d03efe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:12:03.953330 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ace9f-33e4-46bb-ab60-f4fd0d03efe1 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:12:03.953610 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ace9f-33e4-46bb-ab60-f4fd0d03efe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:12:03.953879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ace9f-33e4-46bb-ab60-f4fd0d03efe1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:12:03.954611 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ace9f-33e4-46bb-ab60-f4fd0d03efe1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:12:03.954924 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ace9f-33e4-46bb-ab60-f4fd0d03efe1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:12:03.955955 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: WARNING py.warnings [None req-ee8ace9f-33e4-46bb-ab60-f4fd0d03efe1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 27 21:12:03.955955 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: warnings.warn(msg) Jun 27 21:12:03.955955 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: Jun 27 21:12:03.956843 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ace9f-33e4-46bb-ab60-f4fd0d03efe1 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:12:03.969842 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 755/1496] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:12:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:12:03.978126 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d88774-0afd-42a0-a2de-568f609396f8 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:12:03.987674 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d88774-0afd-42a0-a2de-568f609396f8 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:12:03.992877 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0314aed-8704-4ac4-800d-6ef965065f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:12:03.993566 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0314aed-8704-4ac4-800d-6ef965065f0e admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:12:03.993874 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0314aed-8704-4ac4-800d-6ef965065f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:12:03.994145 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0314aed-8704-4ac4-800d-6ef965065f0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:12:03.994823 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0314aed-8704-4ac4-800d-6ef965065f0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:12:03.996613 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: WARNING py.warnings [None req-c0314aed-8704-4ac4-800d-6ef965065f0e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 21:12:03.996613 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: warnings.warn(msg) Jun 27 21:12:03.996613 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: Jun 27 21:12:03.997864 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0314aed-8704-4ac4-800d-6ef965065f0e admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:12:04.038530 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 742/1497] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:12:03 2022] POST /identity/v3/projects => generated 377 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:12:04.043771 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ace9f-33e4-46bb-ab60-f4fd0d03efe1 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:12:04.054181 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ace9f-33e4-46bb-ab60-f4fd0d03efe1 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:12:04.112461 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e51eaf9-124e-4187-b85d-2c32970b1e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:12:04.113219 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51eaf9-124e-4187-b85d-2c32970b1e23 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:12:04.113436 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51eaf9-124e-4187-b85d-2c32970b1e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:12:04.113706 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51eaf9-124e-4187-b85d-2c32970b1e23 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:12:04.114397 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e51eaf9-124e-4187-b85d-2c32970b1e23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:12:04.115441 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: WARNING py.warnings [None req-5e51eaf9-124e-4187-b85d-2c32970b1e23 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 21:12:04.115441 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: warnings.warn(msg) Jun 27 21:12:04.115441 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: Jun 27 21:12:04.116677 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e51eaf9-124e-4187-b85d-2c32970b1e23 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:12:04.137229 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 756/1498] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:12:04 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:12:04.141463 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0314aed-8704-4ac4-800d-6ef965065f0e admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:12:04.152534 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0314aed-8704-4ac4-800d-6ef965065f0e admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:12:04.158915 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a618ccb-794f-4142-8a2b-1a45b8eb31b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:12:04.159615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a618ccb-794f-4142-8a2b-1a45b8eb31b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:12:04.160200 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a618ccb-794f-4142-8a2b-1a45b8eb31b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:12:04.160525 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a618ccb-794f-4142-8a2b-1a45b8eb31b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:12:04.161501 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a618ccb-794f-4142-8a2b-1a45b8eb31b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:12:04.162539 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: WARNING py.warnings [None req-0a618ccb-794f-4142-8a2b-1a45b8eb31b2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 27 21:12:04.162539 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: warnings.warn(msg) Jun 27 21:12:04.162539 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: Jun 27 21:12:04.163475 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a618ccb-794f-4142-8a2b-1a45b8eb31b2 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:12:04.177580 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 743/1499] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:12:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:12:04.181802 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e51eaf9-124e-4187-b85d-2c32970b1e23 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:12:04.192101 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e51eaf9-124e-4187-b85d-2c32970b1e23 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:12:04.196766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfd0598-1f9b-4995-9778-4374169e67e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:12:04.197468 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd0598-1f9b-4995-9778-4374169e67e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:12:04.197773 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd0598-1f9b-4995-9778-4374169e67e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:12:04.198022 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd0598-1f9b-4995-9778-4374169e67e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:12:04.198754 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfd0598-1f9b-4995-9778-4374169e67e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:12:04.199783 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: WARNING py.warnings [None req-adfd0598-1f9b-4995-9778-4374169e67e7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 27 21:12:04.199783 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: warnings.warn(msg) Jun 27 21:12:04.199783 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: Jun 27 21:12:04.200673 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfd0598-1f9b-4995-9778-4374169e67e7 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:12:04.215180 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 757/1500] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:12:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:12:04.219941 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a618ccb-794f-4142-8a2b-1a45b8eb31b2 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:12:04.230914 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a618ccb-794f-4142-8a2b-1a45b8eb31b2 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:12:04.236801 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db4b12e-048e-4a19-9371-27ac5c12ef3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:12:04.237745 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4b12e-048e-4a19-9371-27ac5c12ef3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:12:04.238023 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4b12e-048e-4a19-9371-27ac5c12ef3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:12:04.238301 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4b12e-048e-4a19-9371-27ac5c12ef3b admin admin] PATH_INFO: `/v3/projects/527f4bc086854c0d91ceb84f39343f0a/users/4b9f579f39bf4f14807f354630681949/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:12:04.255348 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db4b12e-048e-4a19-9371-27ac5c12ef3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=527f4bc086854c0d91ceb84f39343f0a, user_id=4b9f579f39bf4f14807f354630681949, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:12:04.256422 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: WARNING py.warnings [None req-9db4b12e-048e-4a19-9371-27ac5c12ef3b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 21:12:04.256422 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: warnings.warn(msg) Jun 27 21:12:04.256422 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: Jun 27 21:12:04.258520 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db4b12e-048e-4a19-9371-27ac5c12ef3b admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:12:04.273082 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 744/1501] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:12:04 2022] PUT /identity/v3/projects/527f4bc086854c0d91ceb84f39343f0a/users/4b9f579f39bf4f14807f354630681949/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:12:04.284588 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5575b53-2843-4b52-91ed-58c0dec732fe None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:12:04.284882 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5575b53-2843-4b52-91ed-58c0dec732fe None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:12:04.285234 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5575b53-2843-4b52-91ed-58c0dec732fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:12:04.312533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-e5575b53-2843-4b52-91ed-58c0dec732fe None None] MFA Rules not processed for user `4b9f579f39bf4f14807f354630681949`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:12:04.313777 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-e5575b53-2843-4b52-91ed-58c0dec732fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:12:04.379265 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 758/1502] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:12:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:12:04.392754 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3751fed-628c-4b17-b60e-45ad42309a4c None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:12:04.393081 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3751fed-628c-4b17-b60e-45ad42309a4c None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:12:04.393360 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3751fed-628c-4b17-b60e-45ad42309a4c None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:12:04.394731 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 745/1503] 10.222.0.13 () {54 vars in 963 bytes} [Mon Jun 27 21:12:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:12:04.399829 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a76d4-76d9-4a8f-92dd-aef885129be0 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:12:04.400117 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a76d4-76d9-4a8f-92dd-aef885129be0 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:12:04.400389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a76d4-76d9-4a8f-92dd-aef885129be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:12:04.419045 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-929a76d4-76d9-4a8f-92dd-aef885129be0 None None] MFA Rules not processed for user `340915e7debe4ce6828bd8a477ed6f16`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:12:04.420255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-929a76d4-76d9-4a8f-92dd-aef885129be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:12:04.464772 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:12:04.474081 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 759/1504] 10.222.0.13 () {58 vars in 1060 bytes} [Mon Jun 27 21:12:04 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:12:04.481907 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3751fed-628c-4b17-b60e-45ad42309a4c None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:12:04.493384 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3751fed-628c-4b17-b60e-45ad42309a4c None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:12:04.497146 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e676ddf-f865-4be3-b7f3-bff51014ffef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:12:04.497776 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e676ddf-f865-4be3-b7f3-bff51014ffef service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:12:04.498049 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e676ddf-f865-4be3-b7f3-bff51014ffef service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:12:04.498320 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e676ddf-f865-4be3-b7f3-bff51014ffef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:12:04.507125 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e676ddf-f865-4be3-b7f3-bff51014ffef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:12:04.508628 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e676ddf-f865-4be3-b7f3-bff51014ffef service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:12:04.517744 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 746/1505] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:12:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:12:04.560622 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2de2c-6e1f-4497-9fcd-e300f6e62949 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:12:04.560909 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2de2c-6e1f-4497-9fcd-e300f6e62949 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:12:04.561218 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2de2c-6e1f-4497-9fcd-e300f6e62949 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:12:04.562516 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 760/1506] 10.222.0.13 () {52 vars in 829 bytes} [Mon Jun 27 21:12:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:12:04.566355 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e676ddf-f865-4be3-b7f3-bff51014ffef service cinder] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:12:04.632270 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e676ddf-f865-4be3-b7f3-bff51014ffef service cinder] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:12:04.637486 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb1517-b9bb-45ff-ae48-c83263d2587d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:12:04.638356 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb1517-b9bb-45ff-ae48-c83263d2587d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:12:04.638644 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb1517-b9bb-45ff-ae48-c83263d2587d admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:12:04.638921 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb1517-b9bb-45ff-ae48-c83263d2587d admin admin] PATH_INFO: `/v3/users/4b9f579f39bf4f14807f354630681949` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:12:04.640174 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb1517-b9bb-45ff-ae48-c83263d2587d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b9f579f39bf4f14807f354630681949)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:12:04.641236 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: WARNING py.warnings [None req-2aeb1517-b9bb-45ff-ae48-c83263d2587d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 21:12:04.641236 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: warnings.warn(msg) Jun 27 21:12:04.641236 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: Jun 27 21:12:04.642445 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb1517-b9bb-45ff-ae48-c83263d2587d admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:12:04.708788 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-2aeb1517-b9bb-45ff-ae48-c83263d2587d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b9f579f39bf4f14807f354630681949'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:12:04.723955 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-2aeb1517-b9bb-45ff-ae48-c83263d2587d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b9f579f39bf4f14807f354630681949'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:12:04.724739 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-2aeb1517-b9bb-45ff-ae48-c83263d2587d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b9f579f39bf4f14807f354630681949'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:12:04.725584 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-2aeb1517-b9bb-45ff-ae48-c83263d2587d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b9f579f39bf4f14807f354630681949'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:12:04.736386 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-2aeb1517-b9bb-45ff-ae48-c83263d2587d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b9f579f39bf4f14807f354630681949'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:12:04.753340 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:12:04.762775 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 747/1507] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:12:04 2022] DELETE /identity/v3/users/4b9f579f39bf4f14807f354630681949 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:12:04.767193 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1cc487-90cb-4d7b-b44d-965029e9fa8b None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:12:04.767509 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1cc487-90cb-4d7b-b44d-965029e9fa8b None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:12:04.767843 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1cc487-90cb-4d7b-b44d-965029e9fa8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:12:04.787973 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-ce1cc487-90cb-4d7b-b44d-965029e9fa8b None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:12:04.789238 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ce1cc487-90cb-4d7b-b44d-965029e9fa8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:12:04.853299 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 761/1508] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:12:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:12:04.864426 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb1517-b9bb-45ff-ae48-c83263d2587d admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:12:04.866187 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-2aeb1517-b9bb-45ff-ae48-c83263d2587d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:12:04.929291 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb1517-b9bb-45ff-ae48-c83263d2587d admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:12:04.933811 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1b6cef-c4e5-48ec-b550-4c5374cf53de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:12:04.934463 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b6cef-c4e5-48ec-b550-4c5374cf53de service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:12:04.934750 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b6cef-c4e5-48ec-b550-4c5374cf53de service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:12:04.935015 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b6cef-c4e5-48ec-b550-4c5374cf53de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:12:04.944717 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1b6cef-c4e5-48ec-b550-4c5374cf53de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:12:04.946361 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1b6cef-c4e5-48ec-b550-4c5374cf53de service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:12:04.956100 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 748/1509] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:12:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:12:05.321304 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1cc487-90cb-4d7b-b44d-965029e9fa8b None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:12:05.322854 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ce1cc487-90cb-4d7b-b44d-965029e9fa8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:12:05.335342 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1cc487-90cb-4d7b-b44d-965029e9fa8b None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:12:05.341801 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463870fc-932c-4934-b5d3-042e88212685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:12:05.343179 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463870fc-932c-4934-b5d3-042e88212685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:12:05.343484 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463870fc-932c-4934-b5d3-042e88212685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:12:05.343721 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463870fc-932c-4934-b5d3-042e88212685 admin admin] PATH_INFO: `/v3/projects/527f4bc086854c0d91ceb84f39343f0a` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:12:05.345050 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463870fc-932c-4934-b5d3-042e88212685 admin admin] RBAC: Authorizing `identity:delete_project(project_id=527f4bc086854c0d91ceb84f39343f0a)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:12:05.346093 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: WARNING py.warnings [None req-463870fc-932c-4934-b5d3-042e88212685 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 21:12:05.346093 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: warnings.warn(msg) Jun 27 21:12:05.346093 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: Jun 27 21:12:05.347315 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463870fc-932c-4934-b5d3-042e88212685 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:12:05.410847 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-463870fc-932c-4934-b5d3-042e88212685 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '527f4bc086854c0d91ceb84f39343f0a', 'request_id': 'req-463870fc-932c-4934-b5d3-042e88212685'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:12:05.425878 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-463870fc-932c-4934-b5d3-042e88212685 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '527f4bc086854c0d91ceb84f39343f0a', 'request_id': 'req-463870fc-932c-4934-b5d3-042e88212685'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:12:05.435847 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-463870fc-932c-4934-b5d3-042e88212685 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '527f4bc086854c0d91ceb84f39343f0a', 'request_id': 'req-463870fc-932c-4934-b5d3-042e88212685'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:12:05.446724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-463870fc-932c-4934-b5d3-042e88212685 admin admin] The token cache is being invalidate because project 527f4bc086854c0d91ceb84f39343f0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:12:05.446956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-463870fc-932c-4934-b5d3-042e88212685 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:12:05.447594 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-463870fc-932c-4934-b5d3-042e88212685 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:12:05.449303 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 762/1510] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:12:05 2022] DELETE /identity/v3/projects/527f4bc086854c0d91ceb84f39343f0a => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:12:14.951767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:12:15.443610 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:13:41.932371 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df304a-2825-4fdb-be16-f913ac81007a None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:13:41.932997 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df304a-2825-4fdb-be16-f913ac81007a None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:13:41.932997 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df304a-2825-4fdb-be16-f913ac81007a None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:13:41.934432 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 749/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 21:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:13:41.939611 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf451f-c903-46e3-81f7-dc7368cac504 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:13:41.940092 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf451f-c903-46e3-81f7-dc7368cac504 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:13:41.940203 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf451f-c903-46e3-81f7-dc7368cac504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:13:41.959947 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-a5bf451f-c903-46e3-81f7-dc7368cac504 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:13:41.961235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a5bf451f-c903-46e3-81f7-dc7368cac504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:13:42.033206 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 763/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 21:13:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:13:42.191932 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9df304a-2825-4fdb-be16-f913ac81007a None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:13:42.193727 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-e9df304a-2825-4fdb-be16-f913ac81007a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:13:42.278609 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9df304a-2825-4fdb-be16-f913ac81007a None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:13:42.285001 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267b3508-87e8-45db-9633-3a59d86c400d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:13:42.286438 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267b3508-87e8-45db-9633-3a59d86c400d service nova] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:13:42.286728 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267b3508-87e8-45db-9633-3a59d86c400d service nova] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:13:42.287092 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267b3508-87e8-45db-9633-3a59d86c400d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:13:42.298053 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267b3508-87e8-45db-9633-3a59d86c400d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:13:42.299708 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267b3508-87e8-45db-9633-3a59d86c400d service nova] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:13:42.310743 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 750/1513] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:13:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:13:43.162534 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559d53d-a649-4652-a1dd-44ca8d012422 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:13:43.163428 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559d53d-a649-4652-a1dd-44ca8d012422 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:13:43.163428 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559d53d-a649-4652-a1dd-44ca8d012422 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:13:43.165183 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 764/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 21:13:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:13:43.171102 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf99d3e-cb57-4895-9f77-44f7346f21b5 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:13:43.171653 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf99d3e-cb57-4895-9f77-44f7346f21b5 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:13:43.171653 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf99d3e-cb57-4895-9f77-44f7346f21b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:13:43.194993 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-dbf99d3e-cb57-4895-9f77-44f7346f21b5 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:13:43.196255 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-dbf99d3e-cb57-4895-9f77-44f7346f21b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:13:43.204674 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 751/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 21:13:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:13:43.360081 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559d53d-a649-4652-a1dd-44ca8d012422 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:13:43.377202 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559d53d-a649-4652-a1dd-44ca8d012422 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:13:43.383647 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a51776e-cdba-4aca-8c24-a074233a9c31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:13:43.384448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51776e-cdba-4aca-8c24-a074233a9c31 service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:13:43.384830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51776e-cdba-4aca-8c24-a074233a9c31 service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:13:43.385242 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51776e-cdba-4aca-8c24-a074233a9c31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:13:43.396848 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a51776e-cdba-4aca-8c24-a074233a9c31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:13:43.401247 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a51776e-cdba-4aca-8c24-a074233a9c31 service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:13:43.417392 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 765/1516] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:13:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:13:43.443052 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ba67a-5363-4c04-a68d-af749d57af77 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:13:43.443550 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ba67a-5363-4c04-a68d-af749d57af77 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:13:43.443930 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ba67a-5363-4c04-a68d-af749d57af77 None None] PATH_INFO: `/v3` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:13:43.446625 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 752/1517] 10.222.0.33 () {54 vars in 925 bytes} [Mon Jun 27 21:13:43 2022] GET /identity/v3 => generated 255 bytes in 5 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:13:43.454554 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5b5e77-8dd8-4ffb-92b9-04a5acc033b3 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:13:43.455046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5b5e77-8dd8-4ffb-92b9-04a5acc033b3 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:13:43.455698 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5b5e77-8dd8-4ffb-92b9-04a5acc033b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:13:43.487988 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-2d5b5e77-8dd8-4ffb-92b9-04a5acc033b3 None None] MFA Rules not processed for user `0437c45d72cd4a38b9d48ee6f50946fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:13:43.490096 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-2d5b5e77-8dd8-4ffb-92b9-04a5acc033b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:13:43.572818 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 766/1518] 10.222.0.33 () {58 vars in 1015 bytes} [Mon Jun 27 21:13:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:13:52.396683 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a6e0bb-116c-450f-bd63-3a63f020fa62 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:13:52.397279 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a6e0bb-116c-450f-bd63-3a63f020fa62 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:13:52.397279 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a6e0bb-116c-450f-bd63-3a63f020fa62 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:13:52.398536 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 753/1519] 10.222.0.13 () {54 vars in 960 bytes} [Mon Jun 27 21:13:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:13:52.403827 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52e9ae-8d5f-4d9e-a0c2-9bd7c681a968 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:13:52.404300 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52e9ae-8d5f-4d9e-a0c2-9bd7c681a968 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:13:52.404406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52e9ae-8d5f-4d9e-a0c2-9bd7c681a968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:13:52.423937 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-ed52e9ae-8d5f-4d9e-a0c2-9bd7c681a968 None None] MFA Rules not processed for user `0437c45d72cd4a38b9d48ee6f50946fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:13:52.425199 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ed52e9ae-8d5f-4d9e-a0c2-9bd7c681a968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:13:52.435650 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 767/1520] 10.222.0.13 () {58 vars in 1057 bytes} [Mon Jun 27 21:13:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:13:52.443306 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a6e0bb-116c-450f-bd63-3a63f020fa62 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:13:52.458738 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a6e0bb-116c-450f-bd63-3a63f020fa62 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:13:52.464842 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc0a1b2-5e50-41e5-a1d0-17c065db1b2c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0437c45d72cd4a38b9d48ee6f50946fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:13:52.466222 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0a1b2-5e50-41e5-a1d0-17c065db1b2c service placement] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:13:52.466541 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0a1b2-5e50-41e5-a1d0-17c065db1b2c service placement] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:13:52.466819 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0a1b2-5e50-41e5-a1d0-17c065db1b2c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:13:52.476001 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc0a1b2-5e50-41e5-a1d0-17c065db1b2c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:13:52.477659 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc0a1b2-5e50-41e5-a1d0-17c065db1b2c service placement] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:13:52.487931 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 754/1521] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:13:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:13:54.699422 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678aaaa0-488f-4858-9d77-1e878bd4f1bc None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:13:54.700198 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678aaaa0-488f-4858-9d77-1e878bd4f1bc None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:13:54.700198 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678aaaa0-488f-4858-9d77-1e878bd4f1bc None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:13:54.701608 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 768/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 21:13:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:13:54.710584 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d9bcbb-7e0a-4d56-9b47-b924fdc643c5 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:13:54.711334 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d9bcbb-7e0a-4d56-9b47-b924fdc643c5 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:13:54.711334 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d9bcbb-7e0a-4d56-9b47-b924fdc643c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:13:54.732757 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-33d9bcbb-7e0a-4d56-9b47-b924fdc643c5 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:13:54.734017 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-33d9bcbb-7e0a-4d56-9b47-b924fdc643c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:13:54.742295 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 755/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 21:13:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:13:54.896407 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678aaaa0-488f-4858-9d77-1e878bd4f1bc None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:13:54.922404 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678aaaa0-488f-4858-9d77-1e878bd4f1bc None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:13:54.931764 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc5080-7cc4-46b2-8f26-4694d49fc321 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:13:54.932777 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc5080-7cc4-46b2-8f26-4694d49fc321 service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:13:54.933272 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc5080-7cc4-46b2-8f26-4694d49fc321 service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:13:54.933734 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc5080-7cc4-46b2-8f26-4694d49fc321 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:13:54.944281 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc5080-7cc4-46b2-8f26-4694d49fc321 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:13:54.945985 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc5080-7cc4-46b2-8f26-4694d49fc321 service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:13:54.957815 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 769/1524] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:13:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:13:56.370076 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e3337-4bb6-43dd-aa35-265b03701381 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:13:56.370667 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e3337-4bb6-43dd-aa35-265b03701381 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:13:56.370667 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e3337-4bb6-43dd-aa35-265b03701381 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:13:56.371972 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 756/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 27 21:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:13:56.378425 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769d06c-5b51-472c-80aa-cb69b5ed5054 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:13:56.379069 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769d06c-5b51-472c-80aa-cb69b5ed5054 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:13:56.379238 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769d06c-5b51-472c-80aa-cb69b5ed5054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:13:56.399624 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-4769d06c-5b51-472c-80aa-cb69b5ed5054 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:13:56.400893 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-4769d06c-5b51-472c-80aa-cb69b5ed5054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:13:56.409963 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 770/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 27 21:13:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:13:56.566975 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e3337-4bb6-43dd-aa35-265b03701381 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:13:56.582880 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e3337-4bb6-43dd-aa35-265b03701381 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:13:56.588646 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38153f1-72f5-4542-aab7-f56d5ee23498 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:13:56.589351 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38153f1-72f5-4542-aab7-f56d5ee23498 service nova] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:13:56.589637 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38153f1-72f5-4542-aab7-f56d5ee23498 service nova] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:13:56.589911 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38153f1-72f5-4542-aab7-f56d5ee23498 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:13:56.600791 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38153f1-72f5-4542-aab7-f56d5ee23498 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:13:56.602448 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38153f1-72f5-4542-aab7-f56d5ee23498 service nova] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:13:56.613567 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 757/1527] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:13:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:14:00.725755 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc42b48f-3158-431e-a61c-5239a8112924 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:14:00.726670 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc42b48f-3158-431e-a61c-5239a8112924 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:14:00.726670 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc42b48f-3158-431e-a61c-5239a8112924 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:14:00.728370 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 771/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 21:14:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:14:00.734900 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217d389-2f93-419d-abfc-8d64bad128b0 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:14:00.735468 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217d389-2f93-419d-abfc-8d64bad128b0 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:14:00.735468 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217d389-2f93-419d-abfc-8d64bad128b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:14:00.755350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-7217d389-2f93-419d-abfc-8d64bad128b0 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:14:00.756553 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-7217d389-2f93-419d-abfc-8d64bad128b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:14:00.765121 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 758/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 21:14:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:14:00.803086 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc42b48f-3158-431e-a61c-5239a8112924 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:14:00.805157 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-bc42b48f-3158-431e-a61c-5239a8112924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:14:00.868947 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc42b48f-3158-431e-a61c-5239a8112924 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:14:00.872854 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9dce14-cf8d-4df6-92d2-755b00ad597f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:14:00.873533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9dce14-cf8d-4df6-92d2-755b00ad597f service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:14:00.873812 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9dce14-cf8d-4df6-92d2-755b00ad597f service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:14:00.874071 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9dce14-cf8d-4df6-92d2-755b00ad597f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:14:00.883910 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9dce14-cf8d-4df6-92d2-755b00ad597f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:14:00.885421 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9dce14-cf8d-4df6-92d2-755b00ad597f service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:14:00.891565 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:14:00.897861 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 772/1530] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:14:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:14:02.959674 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ead33-56a9-4557-b68e-da555b499f70 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:14:02.960226 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ead33-56a9-4557-b68e-da555b499f70 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:14:02.960322 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ead33-56a9-4557-b68e-da555b499f70 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:14:02.961937 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 759/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 21:14:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:14:02.969639 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef451f74-dee2-47dd-8c4c-3b902f6f35ee None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:14:02.970187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef451f74-dee2-47dd-8c4c-3b902f6f35ee None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:14:02.970597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef451f74-dee2-47dd-8c4c-3b902f6f35ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:14:03.000915 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-ef451f74-dee2-47dd-8c4c-3b902f6f35ee None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:14:03.002184 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ef451f74-dee2-47dd-8c4c-3b902f6f35ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:14:03.010893 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 773/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 21:14:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:14:03.050328 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ead33-56a9-4557-b68e-da555b499f70 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:14:03.052396 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-280ead33-56a9-4557-b68e-da555b499f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:14:03.066997 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ead33-56a9-4557-b68e-da555b499f70 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:14:03.070981 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58229603-31ed-4ae6-868c-4ba80dcda413 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:14:03.071653 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58229603-31ed-4ae6-868c-4ba80dcda413 service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:14:03.071941 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58229603-31ed-4ae6-868c-4ba80dcda413 service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:14:03.072217 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58229603-31ed-4ae6-868c-4ba80dcda413 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:14:03.082822 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58229603-31ed-4ae6-868c-4ba80dcda413 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:14:03.084281 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58229603-31ed-4ae6-868c-4ba80dcda413 service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:14:03.095647 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 760/1533] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:14:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:14:06.804063 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b77f2a-1d21-4510-9e19-4e1f9df43ce9 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:14:06.804722 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b77f2a-1d21-4510-9e19-4e1f9df43ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:14:06.804722 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b77f2a-1d21-4510-9e19-4e1f9df43ce9 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:14:06.806153 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 774/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 21:14:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:14:06.812632 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dafd809-13bd-484c-ac77-eb165854cdcb None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:14:06.813260 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dafd809-13bd-484c-ac77-eb165854cdcb None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:14:06.813260 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dafd809-13bd-484c-ac77-eb165854cdcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:14:06.820543 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=62 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:14:06.836396 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-5dafd809-13bd-484c-ac77-eb165854cdcb None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:14:06.837680 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-5dafd809-13bd-484c-ac77-eb165854cdcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:14:06.847427 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 761/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 21:14:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:14:07.159626 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b77f2a-1d21-4510-9e19-4e1f9df43ce9 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:14:07.161728 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-53b77f2a-1d21-4510-9e19-4e1f9df43ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:14:07.232047 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b77f2a-1d21-4510-9e19-4e1f9df43ce9 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:14:07.236233 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5494a0fe-7c4c-4805-81c0-6e5e40b52e5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:14:07.236972 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494a0fe-7c4c-4805-81c0-6e5e40b52e5e service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:14:07.237254 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494a0fe-7c4c-4805-81c0-6e5e40b52e5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:14:07.237522 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494a0fe-7c4c-4805-81c0-6e5e40b52e5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:14:07.249048 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5494a0fe-7c4c-4805-81c0-6e5e40b52e5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:14:07.250930 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5494a0fe-7c4c-4805-81c0-6e5e40b52e5e service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:14:07.262517 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 775/1536] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:14:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:14:10.129359 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34b220-ab61-45e4-bf95-a46c7031f7cd None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:14:10.129954 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34b220-ab61-45e4-bf95-a46c7031f7cd None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:14:10.129954 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34b220-ab61-45e4-bf95-a46c7031f7cd None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:14:10.131218 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 762/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 27 21:14:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:14:10.137211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acfc93f-7f67-48f5-9008-81beaaa122d1 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:14:10.138089 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acfc93f-7f67-48f5-9008-81beaaa122d1 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:14:10.138089 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acfc93f-7f67-48f5-9008-81beaaa122d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:14:10.158765 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-5acfc93f-7f67-48f5-9008-81beaaa122d1 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:14:10.160037 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-5acfc93f-7f67-48f5-9008-81beaaa122d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:14:10.169430 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 776/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 27 21:14:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:14:10.640001 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d34b220-ab61-45e4-bf95-a46c7031f7cd None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:14:10.641789 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-1d34b220-ab61-45e4-bf95-a46c7031f7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:14:10.655473 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d34b220-ab61-45e4-bf95-a46c7031f7cd None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:14:10.659168 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e49504-1b86-4923-90e4-c76a5745cc2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:14:10.659823 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e49504-1b86-4923-90e4-c76a5745cc2d service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:14:10.660101 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e49504-1b86-4923-90e4-c76a5745cc2d service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:14:10.660374 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e49504-1b86-4923-90e4-c76a5745cc2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:14:10.670655 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e49504-1b86-4923-90e4-c76a5745cc2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:14:10.672166 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e49504-1b86-4923-90e4-c76a5745cc2d service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:14:10.682360 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 763/1539] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:14:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:14:20.152763 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=32 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:14:20.677801 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=8 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:41.373478 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5a0da-1dc1-46c4-b502-8d311d029262 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:41.374443 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5a0da-1dc1-46c4-b502-8d311d029262 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:41.374443 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5a0da-1dc1-46c4-b502-8d311d029262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:41.407593 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-60c5a0da-1dc1-46c4-b502-8d311d029262 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:41.409010 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-60c5a0da-1dc1-46c4-b502-8d311d029262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:41.424670 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 777/1540] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:15:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:15:41.441367 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83210b1-5ec7-4f18-a2f3-78b89b76c2ad None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:41.442956 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83210b1-5ec7-4f18-a2f3-78b89b76c2ad None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:41.443869 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83210b1-5ec7-4f18-a2f3-78b89b76c2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:41.496959 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-d83210b1-5ec7-4f18-a2f3-78b89b76c2ad None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:41.500184 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-d83210b1-5ec7-4f18-a2f3-78b89b76c2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:41.524951 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 764/1541] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:15:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:15:41.530109 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5a0da-1dc1-46c4-b502-8d311d029262 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:41.545632 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5a0da-1dc1-46c4-b502-8d311d029262 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:41.555039 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b9851-db02-4df6-b1c9-58f7f466e290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:41.556852 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b9851-db02-4df6-b1c9-58f7f466e290 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:41.556852 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b9851-db02-4df6-b1c9-58f7f466e290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:41.556852 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b9851-db02-4df6-b1c9-58f7f466e290 admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:41.557745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b9851-db02-4df6-b1c9-58f7f466e290 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:15:41.558229 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b9851-db02-4df6-b1c9-58f7f466e290 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:41.559947 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b9851-db02-4df6-b1c9-58f7f466e290 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:41.579694 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 778/1542] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:15:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:15:41.598398 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b9851-db02-4df6-b1c9-58f7f466e290 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:41.600066 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fceda4b-2c49-4682-a6d1-ab2f8b72ba9a None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:41.600563 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fceda4b-2c49-4682-a6d1-ab2f8b72ba9a None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:41.601277 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fceda4b-2c49-4682-a6d1-ab2f8b72ba9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:41.614649 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b9851-db02-4df6-b1c9-58f7f466e290 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:41.625022 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4427e19-c373-4708-92e1-69c32fe2506a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:41.625621 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4427e19-c373-4708-92e1-69c32fe2506a admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:41.625900 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4427e19-c373-4708-92e1-69c32fe2506a admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:41.626268 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4427e19-c373-4708-92e1-69c32fe2506a admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:41.626883 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4427e19-c373-4708-92e1-69c32fe2506a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:41.627936 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: WARNING py.warnings [None req-b4427e19-c373-4708-92e1-69c32fe2506a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 21:15:41.627936 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: warnings.warn(msg) Jun 27 21:15:41.627936 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: Jun 27 21:15:41.629203 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4427e19-c373-4708-92e1-69c32fe2506a admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:41.634460 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-9fceda4b-2c49-4682-a6d1-ab2f8b72ba9a None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:41.636692 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-9fceda4b-2c49-4682-a6d1-ab2f8b72ba9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:41.663049 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 765/1543] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:15:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:41.683356 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 779/1544] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:15:41 2022] POST /identity/v3/projects => generated 393 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:15:41.684666 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d706eb6-33dc-4ac0-89a6-7f3a11f0b4e3 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:41.685244 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d706eb6-33dc-4ac0-89a6-7f3a11f0b4e3 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:41.685957 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d706eb6-33dc-4ac0-89a6-7f3a11f0b4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:41.694166 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4427e19-c373-4708-92e1-69c32fe2506a admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:41.739083 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-6d706eb6-33dc-4ac0-89a6-7f3a11f0b4e3 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:41.741515 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-6d706eb6-33dc-4ac0-89a6-7f3a11f0b4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:41.821487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4427e19-c373-4708-92e1-69c32fe2506a admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:41.835106 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23108448-bc45-403b-8b0c-18921198542a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:41.836007 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23108448-bc45-403b-8b0c-18921198542a admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:41.836300 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23108448-bc45-403b-8b0c-18921198542a admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:41.836575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23108448-bc45-403b-8b0c-18921198542a admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:41.837339 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23108448-bc45-403b-8b0c-18921198542a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:41.840387 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23108448-bc45-403b-8b0c-18921198542a admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:41.898892 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 780/1545] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:15:41 2022] POST /identity/v3/users => generated 297 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:15:41.900183 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566126f2-7201-4397-81b1-4834e9a24980 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:41.900468 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566126f2-7201-4397-81b1-4834e9a24980 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:41.900953 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566126f2-7201-4397-81b1-4834e9a24980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:41.941751 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 766/1546] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:15:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:41.943313 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d706eb6-33dc-4ac0-89a6-7f3a11f0b4e3 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:41.946605 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-566126f2-7201-4397-81b1-4834e9a24980 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:41.954621 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-566126f2-7201-4397-81b1-4834e9a24980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:41.981395 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d706eb6-33dc-4ac0-89a6-7f3a11f0b4e3 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:41.985024 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 781/1547] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:15:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:41.987962 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566126f2-7201-4397-81b1-4834e9a24980 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:42.000601 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d94c9e-c8e8-4987-af22-1aa4f74dba3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:42.001809 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d94c9e-c8e8-4987-af22-1aa4f74dba3e admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:42.002279 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d94c9e-c8e8-4987-af22-1aa4f74dba3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:42.002776 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d94c9e-c8e8-4987-af22-1aa4f74dba3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:42.004032 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d94c9e-c8e8-4987-af22-1aa4f74dba3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:42.007922 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d94c9e-c8e8-4987-af22-1aa4f74dba3e admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:42.016297 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566126f2-7201-4397-81b1-4834e9a24980 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:42.051204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59a9e95-8843-4227-a09b-f0eeae0ac0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:42.055264 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a9e95-8843-4227-a09b-f0eeae0ac0f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:42.055754 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a9e95-8843-4227-a09b-f0eeae0ac0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:42.056257 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a9e95-8843-4227-a09b-f0eeae0ac0f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:42.057775 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a9e95-8843-4227-a09b-f0eeae0ac0f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:15:42.058329 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a9e95-8843-4227-a09b-f0eeae0ac0f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:42.059785 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 767/1548] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:41 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:42.062695 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a4833-74e6-4ce1-a879-d8ed76ee20c6 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:42.063206 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a4833-74e6-4ce1-a879-d8ed76ee20c6 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:42.063687 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a4833-74e6-4ce1-a879-d8ed76ee20c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:42.068129 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a9e95-8843-4227-a09b-f0eeae0ac0f2 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:42.095879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 782/1549] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:15:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:15:42.098186 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56982cdf-3d90-4a24-9e4b-615c15a864b7 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:42.098684 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56982cdf-3d90-4a24-9e4b-615c15a864b7 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:42.099161 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56982cdf-3d90-4a24-9e4b-615c15a864b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:42.112875 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-0f8a4833-74e6-4ce1-a879-d8ed76ee20c6 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:42.115006 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-0f8a4833-74e6-4ce1-a879-d8ed76ee20c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:42.124679 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 768/1550] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:42.126499 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f393301-27cf-4208-86c3-33ab336ed66d None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:42.127285 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f393301-27cf-4208-86c3-33ab336ed66d None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:42.127835 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f393301-27cf-4208-86c3-33ab336ed66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:42.137803 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-56982cdf-3d90-4a24-9e4b-615c15a864b7 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:42.139898 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-56982cdf-3d90-4a24-9e4b-615c15a864b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:42.149385 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-9f393301-27cf-4208-86c3-33ab336ed66d None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:42.150544 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 783/1551] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:42.151410 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-9f393301-27cf-4208-86c3-33ab336ed66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:42.156320 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56982cdf-3d90-4a24-9e4b-615c15a864b7 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:42.160151 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 769/1552] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:42.168630 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae61b9-c09b-44f2-bb7d-bd8d94849da1 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:42.168977 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae61b9-c09b-44f2-bb7d-bd8d94849da1 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:42.169386 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae61b9-c09b-44f2-bb7d-bd8d94849da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:42.170238 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56982cdf-3d90-4a24-9e4b-615c15a864b7 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:42.176548 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2258fc9c-db2b-4bdf-8cf5-653adfc3670d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:42.178452 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2258fc9c-db2b-4bdf-8cf5-653adfc3670d admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:42.178901 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2258fc9c-db2b-4bdf-8cf5-653adfc3670d admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:42.179324 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2258fc9c-db2b-4bdf-8cf5-653adfc3670d admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:42.180226 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2258fc9c-db2b-4bdf-8cf5-653adfc3670d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:42.182331 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2258fc9c-db2b-4bdf-8cf5-653adfc3670d admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:42.188700 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-17ae61b9-c09b-44f2-bb7d-bd8d94849da1 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:42.190005 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-17ae61b9-c09b-44f2-bb7d-bd8d94849da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:42.202483 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 784/1553] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:42.202786 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 770/1554] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:42.202786 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f443b-13b6-425a-8897-786f4466dfa0 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:42.202786 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f443b-13b6-425a-8897-786f4466dfa0 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:42.202786 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f443b-13b6-425a-8897-786f4466dfa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:42.203343 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2258fc9c-db2b-4bdf-8cf5-653adfc3670d admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:42.222951 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2258fc9c-db2b-4bdf-8cf5-653adfc3670d admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:42.224833 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-395f443b-13b6-425a-8897-786f4466dfa0 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:42.226186 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-395f443b-13b6-425a-8897-786f4466dfa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:42.229568 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea160cc-4569-4058-be58-77b7001b2a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:42.230486 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea160cc-4569-4058-be58-77b7001b2a30 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:42.230941 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea160cc-4569-4058-be58-77b7001b2a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:42.231290 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea160cc-4569-4058-be58-77b7001b2a30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:42.232132 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea160cc-4569-4058-be58-77b7001b2a30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:42.234672 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea160cc-4569-4058-be58-77b7001b2a30 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:42.235826 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 771/1555] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:42.243193 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ed98f-5849-422b-9080-a9ce73d4eff7 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:42.243660 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ed98f-5849-422b-9080-a9ce73d4eff7 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:42.244337 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ed98f-5849-422b-9080-a9ce73d4eff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:42.277819 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-3d4ed98f-5849-422b-9080-a9ce73d4eff7 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:42.279862 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-3d4ed98f-5849-422b-9080-a9ce73d4eff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:42.286694 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 785/1556] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:15:42 2022] POST /identity/v3/projects => generated 383 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:42.288343 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea160cc-4569-4058-be58-77b7001b2a30 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:42.291700 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 772/1557] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:42.293295 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403cbb8f-41a8-4aba-9510-fc32bd888ba8 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:42.293590 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403cbb8f-41a8-4aba-9510-fc32bd888ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:42.293872 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403cbb8f-41a8-4aba-9510-fc32bd888ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:42.314007 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-403cbb8f-41a8-4aba-9510-fc32bd888ba8 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:42.315258 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-403cbb8f-41a8-4aba-9510-fc32bd888ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:42.347441 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:42.366398 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea160cc-4569-4058-be58-77b7001b2a30 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:42.371177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4167b88-16aa-4872-979d-fb3e532f119b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:42.371859 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4167b88-16aa-4872-979d-fb3e532f119b admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:42.372143 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4167b88-16aa-4872-979d-fb3e532f119b admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:42.372419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4167b88-16aa-4872-979d-fb3e532f119b admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:42.373329 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4167b88-16aa-4872-979d-fb3e532f119b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:15:42.373646 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4167b88-16aa-4872-979d-fb3e532f119b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:42.375296 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4167b88-16aa-4872-979d-fb3e532f119b admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:42.385962 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 773/1558] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:42.389974 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403cbb8f-41a8-4aba-9510-fc32bd888ba8 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:42.395018 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 786/1559] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:15:42.396895 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb131eb-fdea-47e8-bd9a-a0d55fa416bf None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:42.397238 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb131eb-fdea-47e8-bd9a-a0d55fa416bf None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:42.397521 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb131eb-fdea-47e8-bd9a-a0d55fa416bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:42.404014 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403cbb8f-41a8-4aba-9510-fc32bd888ba8 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:42.411229 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5b246-3eb9-498a-b2d4-3ad53f90004c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:42.412360 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5b246-3eb9-498a-b2d4-3ad53f90004c admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:42.412749 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5b246-3eb9-498a-b2d4-3ad53f90004c admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:42.413173 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5b246-3eb9-498a-b2d4-3ad53f90004c admin admin] PATH_INFO: `/v3/projects/a2a3853bab51428eb89ff228e5f29ef4/users/a0408ee5961249d1bd296a6ef1a030a9/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:42.418332 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-ccb131eb-fdea-47e8-bd9a-a0d55fa416bf None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:42.418961 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ccb131eb-fdea-47e8-bd9a-a0d55fa416bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:42.427574 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 787/1560] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:42.428898 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f0fa0-7792-4178-8252-67ea0689639c None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:42.429221 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f0fa0-7792-4178-8252-67ea0689639c None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:42.429493 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f0fa0-7792-4178-8252-67ea0689639c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:42.438523 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a5b246-3eb9-498a-b2d4-3ad53f90004c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2a3853bab51428eb89ff228e5f29ef4, user_id=a0408ee5961249d1bd296a6ef1a030a9, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:42.441595 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a5b246-3eb9-498a-b2d4-3ad53f90004c admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:42.449073 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-082f0fa0-7792-4178-8252-67ea0689639c None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:42.450282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-082f0fa0-7792-4178-8252-67ea0689639c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:42.459943 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 774/1561] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:15:42 2022] PUT /identity/v3/projects/a2a3853bab51428eb89ff228e5f29ef4/users/a0408ee5961249d1bd296a6ef1a030a9/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:15:42.462392 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb091c13-8632-4b0c-ba75-12b67e4afb61 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:42.462689 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb091c13-8632-4b0c-ba75-12b67e4afb61 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:42.462972 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb091c13-8632-4b0c-ba75-12b67e4afb61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:42.493073 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-bb091c13-8632-4b0c-ba75-12b67e4afb61 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:42.494385 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-bb091c13-8632-4b0c-ba75-12b67e4afb61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:42.511818 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:42.536121 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 788/1562] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:42.537013 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f0fa0-7792-4178-8252-67ea0689639c None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:42.550877 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f0fa0-7792-4178-8252-67ea0689639c None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:42.555888 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbe9737-af78-46a3-a098-e447dcba8bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:42.556568 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbe9737-af78-46a3-a098-e447dcba8bc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:42.556968 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbe9737-af78-46a3-a098-e447dcba8bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:42.557392 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbe9737-af78-46a3-a098-e447dcba8bc6 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:42.558105 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbe9737-af78-46a3-a098-e447dcba8bc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:42.560070 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbe9737-af78-46a3-a098-e447dcba8bc6 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:42.574695 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 775/1563] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:42.575963 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb091c13-8632-4b0c-ba75-12b67e4afb61 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:42.586743 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb091c13-8632-4b0c-ba75-12b67e4afb61 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:42.589849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 789/1564] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:15:42 2022] POST /identity/v3/users => generated 292 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:42.591005 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbe9737-af78-46a3-a098-e447dcba8bc6 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:42.592494 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd568e1-dee1-4cf0-a283-09c8ab9bb73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:42.593218 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd568e1-dee1-4cf0-a283-09c8ab9bb73b admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:42.593507 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd568e1-dee1-4cf0-a283-09c8ab9bb73b admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:42.593785 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd568e1-dee1-4cf0-a283-09c8ab9bb73b admin admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:42.594529 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd568e1-dee1-4cf0-a283-09c8ab9bb73b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:15:42.594840 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd568e1-dee1-4cf0-a283-09c8ab9bb73b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:42.595887 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: WARNING py.warnings [None req-ddd568e1-dee1-4cf0-a283-09c8ab9bb73b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 27 21:15:42.595887 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: warnings.warn(msg) Jun 27 21:15:42.595887 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: Jun 27 21:15:42.596780 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd568e1-dee1-4cf0-a283-09c8ab9bb73b admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:42.601998 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbe9737-af78-46a3-a098-e447dcba8bc6 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:42.607581 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f303421-6875-498e-90fd-e9263682e099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:42.608251 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f303421-6875-498e-90fd-e9263682e099 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:42.608531 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f303421-6875-498e-90fd-e9263682e099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:42.608801 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f303421-6875-498e-90fd-e9263682e099 admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:42.618112 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f303421-6875-498e-90fd-e9263682e099 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:15:42.618412 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 776/1565] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:15:42.618627 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f303421-6875-498e-90fd-e9263682e099 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:42.618835 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd568e1-dee1-4cf0-a283-09c8ab9bb73b admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:42.619023 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f303421-6875-498e-90fd-e9263682e099 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:42.633314 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 790/1566] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:15:42.643214 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f303421-6875-498e-90fd-e9263682e099 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:42.644459 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd568e1-dee1-4cf0-a283-09c8ab9bb73b admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:42.648840 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c3adc9-3ea4-4a79-bee3-b60119fd3ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:42.649079 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f303421-6875-498e-90fd-e9263682e099 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:42.649588 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c3adc9-3ea4-4a79-bee3-b60119fd3ded admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:42.649865 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c3adc9-3ea4-4a79-bee3-b60119fd3ded admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:42.650160 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c3adc9-3ea4-4a79-bee3-b60119fd3ded admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:42.651624 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c3adc9-3ea4-4a79-bee3-b60119fd3ded admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:42.653834 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c3adc9-3ea4-4a79-bee3-b60119fd3ded admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:42.655136 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff71ff9-0705-49e2-bc67-ea624e8147b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:42.656064 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff71ff9-0705-49e2-bc67-ea624e8147b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:42.656547 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff71ff9-0705-49e2-bc67-ea624e8147b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:42.657088 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff71ff9-0705-49e2-bc67-ea624e8147b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:42.658056 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff71ff9-0705-49e2-bc67-ea624e8147b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:15:42.658437 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff71ff9-0705-49e2-bc67-ea624e8147b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:42.660577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff71ff9-0705-49e2-bc67-ea624e8147b3 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:42.681351 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 791/1567] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:15:42.681351 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816cb00-63cc-4e35-97ee-7b49bc40a3f8 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:42.681351 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816cb00-63cc-4e35-97ee-7b49bc40a3f8 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:42.681351 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816cb00-63cc-4e35-97ee-7b49bc40a3f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:42.696060 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 777/1568] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:15:42 2022] POST /identity/v3/projects => generated 365 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:42.697664 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c3adc9-3ea4-4a79-bee3-b60119fd3ded admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:42.703380 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-5816cb00-63cc-4e35-97ee-7b49bc40a3f8 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:42.705235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-5816cb00-63cc-4e35-97ee-7b49bc40a3f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:42.781882 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c3adc9-3ea4-4a79-bee3-b60119fd3ded admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:42.786998 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b96ba-8af8-4d11-bb1a-0db5cb41dc15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:42.787703 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b96ba-8af8-4d11-bb1a-0db5cb41dc15 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:42.787990 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b96ba-8af8-4d11-bb1a-0db5cb41dc15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:42.788264 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b96ba-8af8-4d11-bb1a-0db5cb41dc15 admin admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:42.789466 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b96ba-8af8-4d11-bb1a-0db5cb41dc15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:15:42.789658 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b96ba-8af8-4d11-bb1a-0db5cb41dc15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:42.791449 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b96ba-8af8-4d11-bb1a-0db5cb41dc15 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:42.794082 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 792/1569] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:42.796246 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be788eb6-9b4a-43ec-b317-578624835872 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:42.796639 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be788eb6-9b4a-43ec-b317-578624835872 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:42.796812 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be788eb6-9b4a-43ec-b317-578624835872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:42.807092 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 778/1570] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:15:42.810043 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b96ba-8af8-4d11-bb1a-0db5cb41dc15 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:42.821846 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b96ba-8af8-4d11-bb1a-0db5cb41dc15 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:42.822296 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-be788eb6-9b4a-43ec-b317-578624835872 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:42.824158 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-be788eb6-9b4a-43ec-b317-578624835872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:42.827535 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee7d08a-1a35-46e6-8c7c-09b05df0503b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:42.828238 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7d08a-1a35-46e6-8c7c-09b05df0503b admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:42.828521 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7d08a-1a35-46e6-8c7c-09b05df0503b admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:42.828798 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7d08a-1a35-46e6-8c7c-09b05df0503b admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:42.829631 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee7d08a-1a35-46e6-8c7c-09b05df0503b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:42.831481 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee7d08a-1a35-46e6-8c7c-09b05df0503b admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:42.834192 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 793/1571] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:42.838762 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be788eb6-9b4a-43ec-b317-578624835872 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:42.845305 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 779/1572] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:42.845965 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee7d08a-1a35-46e6-8c7c-09b05df0503b admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:42.853491 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be788eb6-9b4a-43ec-b317-578624835872 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:42.856226 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee7d08a-1a35-46e6-8c7c-09b05df0503b admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:42.859077 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e497fff9-29c1-47ec-a22c-14a03eeeff5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:42.859752 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497fff9-29c1-47ec-a22c-14a03eeeff5c admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:42.860029 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497fff9-29c1-47ec-a22c-14a03eeeff5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:42.860299 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497fff9-29c1-47ec-a22c-14a03eeeff5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:42.861053 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e497fff9-29c1-47ec-a22c-14a03eeeff5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:42.861239 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c82992-ec28-459f-98a1-466071d1c0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:42.861718 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c82992-ec28-459f-98a1-466071d1c0eb admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:42.862002 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c82992-ec28-459f-98a1-466071d1c0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:42.862283 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c82992-ec28-459f-98a1-466071d1c0eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:42.862989 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c82992-ec28-459f-98a1-466071d1c0eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:42.863133 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e497fff9-29c1-47ec-a22c-14a03eeeff5c admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:42.864887 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c82992-ec28-459f-98a1-466071d1c0eb admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:42.898792 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 794/1573] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:15:42 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:42.899104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 780/1574] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:15:42 2022] POST /identity/v3/projects => generated 355 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:42.899104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c82992-ec28-459f-98a1-466071d1c0eb admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:42.899358 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e497fff9-29c1-47ec-a22c-14a03eeeff5c admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:42.905675 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:42.955903 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:42.971011 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e497fff9-29c1-47ec-a22c-14a03eeeff5c admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:42.975957 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726e3b4-3d92-4b6d-ab7c-ae7b0a957093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:42.976168 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c82992-ec28-459f-98a1-466071d1c0eb admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:42.976643 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726e3b4-3d92-4b6d-ab7c-ae7b0a957093 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:42.976948 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726e3b4-3d92-4b6d-ab7c-ae7b0a957093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:42.977233 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726e3b4-3d92-4b6d-ab7c-ae7b0a957093 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:42.977941 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4726e3b4-3d92-4b6d-ab7c-ae7b0a957093 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:42.980283 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4726e3b4-3d92-4b6d-ab7c-ae7b0a957093 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:42.982625 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c392b5dd-7e8b-43a0-b85f-4c1aee875711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:42.983463 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392b5dd-7e8b-43a0-b85f-4c1aee875711 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:42.983895 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392b5dd-7e8b-43a0-b85f-4c1aee875711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:42.984266 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392b5dd-7e8b-43a0-b85f-4c1aee875711 admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:42.985286 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c392b5dd-7e8b-43a0-b85f-4c1aee875711 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:42.986383 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: WARNING py.warnings [None req-c392b5dd-7e8b-43a0-b85f-4c1aee875711 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 21:15:42.986383 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: warnings.warn(msg) Jun 27 21:15:42.986383 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: Jun 27 21:15:42.987852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c392b5dd-7e8b-43a0-b85f-4c1aee875711 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:43.015117 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 795/1575] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:15:42 2022] POST /identity/v3/projects => generated 367 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:43.015117 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726e3b4-3d92-4b6d-ab7c-ae7b0a957093 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:43.016289 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 781/1576] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:15:42 2022] POST /identity/v3/users => generated 283 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:43.017623 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c392b5dd-7e8b-43a0-b85f-4c1aee875711 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:43.032450 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c392b5dd-7e8b-43a0-b85f-4c1aee875711 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:43.083664 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726e3b4-3d92-4b6d-ab7c-ae7b0a957093 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:43.089848 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c9578-8e68-4682-9174-63dfbaa109a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:43.090710 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c9578-8e68-4682-9174-63dfbaa109a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:43.091096 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c9578-8e68-4682-9174-63dfbaa109a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:43.091479 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c9578-8e68-4682-9174-63dfbaa109a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:43.099863 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d550c-04bf-451f-9864-81669d2f9472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:43.100539 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d550c-04bf-451f-9864-81669d2f9472 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:43.100816 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d550c-04bf-451f-9864-81669d2f9472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:43.101151 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d550c-04bf-451f-9864-81669d2f9472 admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:43.101869 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6d550c-04bf-451f-9864-81669d2f9472 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:43.104268 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6d550c-04bf-451f-9864-81669d2f9472 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:43.133531 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 782/1577] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:15:43 2022] POST /identity/v3/projects => generated 379 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:43.134422 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d550c-04bf-451f-9864-81669d2f9472 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:43.165975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65c9578-8e68-4682-9174-63dfbaa109a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:43.167690 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65c9578-8e68-4682-9174-63dfbaa109a3 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:43.179763 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 796/1578] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:15:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:15:43.180468 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c9578-8e68-4682-9174-63dfbaa109a3 service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:43.191169 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c9578-8e68-4682-9174-63dfbaa109a3 service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:43.232827 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d550c-04bf-451f-9864-81669d2f9472 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:43.239424 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:43.239820 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89704eeb-4a1a-4bff-8f92-03511abad019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:43.240705 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89704eeb-4a1a-4bff-8f92-03511abad019 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:43.241143 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89704eeb-4a1a-4bff-8f92-03511abad019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:43.241610 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89704eeb-4a1a-4bff-8f92-03511abad019 admin admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:43.242668 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89704eeb-4a1a-4bff-8f92-03511abad019 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:15:43.243034 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89704eeb-4a1a-4bff-8f92-03511abad019 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:43.245932 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89704eeb-4a1a-4bff-8f92-03511abad019 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:43.274490 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 783/1579] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:15:43.274490 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89704eeb-4a1a-4bff-8f92-03511abad019 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:43.276364 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f5f78-6a90-40a8-ad5c-b230a169377d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:43.277943 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f5f78-6a90-40a8-ad5c-b230a169377d admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:43.278324 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f5f78-6a90-40a8-ad5c-b230a169377d admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:43.278687 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f5f78-6a90-40a8-ad5c-b230a169377d admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:43.279513 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f5f78-6a90-40a8-ad5c-b230a169377d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:43.281543 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f5f78-6a90-40a8-ad5c-b230a169377d admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:43.297992 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 797/1580] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:43.298205 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89704eeb-4a1a-4bff-8f92-03511abad019 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:43.298325 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f5f78-6a90-40a8-ad5c-b230a169377d admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:43.306028 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e395d4f5-a139-43cf-b382-58b78ac9eee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:43.307247 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395d4f5-a139-43cf-b382-58b78ac9eee1 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:43.307742 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395d4f5-a139-43cf-b382-58b78ac9eee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:43.308461 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395d4f5-a139-43cf-b382-58b78ac9eee1 admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:43.309807 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e395d4f5-a139-43cf-b382-58b78ac9eee1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:43.310212 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f5f78-6a90-40a8-ad5c-b230a169377d admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:43.313617 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e395d4f5-a139-43cf-b382-58b78ac9eee1 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:43.316194 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dd8f76-688d-4033-9c07-7cde1f63c88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:43.317045 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd8f76-688d-4033-9c07-7cde1f63c88f admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:43.317419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd8f76-688d-4033-9c07-7cde1f63c88f admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:43.317759 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd8f76-688d-4033-9c07-7cde1f63c88f admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:43.318584 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dd8f76-688d-4033-9c07-7cde1f63c88f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:43.321053 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dd8f76-688d-4033-9c07-7cde1f63c88f admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:43.350076 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 798/1581] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:15:43 2022] POST /identity/v3/users => generated 278 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:43.350076 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dd8f76-688d-4033-9c07-7cde1f63c88f admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:43.350361 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 784/1582] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:15:43 2022] POST /identity/v3/users => generated 288 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:43.351499 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e395d4f5-a139-43cf-b382-58b78ac9eee1 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:43.363211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dd8f76-688d-4033-9c07-7cde1f63c88f admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:43.368457 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3635e-cbe8-408a-b4bc-c9fbd2f9ba20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:43.369182 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3635e-cbe8-408a-b4bc-c9fbd2f9ba20 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:43.369320 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e395d4f5-a139-43cf-b382-58b78ac9eee1 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:43.369947 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3635e-cbe8-408a-b4bc-c9fbd2f9ba20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:43.370434 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3635e-cbe8-408a-b4bc-c9fbd2f9ba20 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:43.371187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a3635e-cbe8-408a-b4bc-c9fbd2f9ba20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:43.373365 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a3635e-cbe8-408a-b4bc-c9fbd2f9ba20 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:43.375023 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23a8df-1f61-4a0d-9dd1-9222d30d678c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:43.375977 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23a8df-1f61-4a0d-9dd1-9222d30d678c admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:43.376366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23a8df-1f61-4a0d-9dd1-9222d30d678c admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:43.376762 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23a8df-1f61-4a0d-9dd1-9222d30d678c admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:43.377662 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd23a8df-1f61-4a0d-9dd1-9222d30d678c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:43.379575 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd23a8df-1f61-4a0d-9dd1-9222d30d678c admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:43.397935 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 785/1583] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:43.399470 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23a8df-1f61-4a0d-9dd1-9222d30d678c admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:43.400302 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 799/1584] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:15:43 2022] POST /identity/v3/users => generated 284 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:43.401881 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3635e-cbe8-408a-b4bc-c9fbd2f9ba20 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:43.414400 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3635e-cbe8-408a-b4bc-c9fbd2f9ba20 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:43.418944 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23a8df-1f61-4a0d-9dd1-9222d30d678c admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:43.420527 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dcb364-370e-4e07-9a1f-b4d078993f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:43.421344 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcb364-370e-4e07-9a1f-b4d078993f46 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:43.421633 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcb364-370e-4e07-9a1f-b4d078993f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:43.421919 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcb364-370e-4e07-9a1f-b4d078993f46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:43.422603 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dcb364-370e-4e07-9a1f-b4d078993f46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:43.424554 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dcb364-370e-4e07-9a1f-b4d078993f46 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:43.425899 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48282f06-b674-4b47-8119-065948743dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:43.426849 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48282f06-b674-4b47-8119-065948743dcd admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:43.427514 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48282f06-b674-4b47-8119-065948743dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:43.427997 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48282f06-b674-4b47-8119-065948743dcd admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:43.429152 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48282f06-b674-4b47-8119-065948743dcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:43.432364 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48282f06-b674-4b47-8119-065948743dcd admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:43.462162 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 800/1585] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:15:43 2022] POST /identity/v3/projects => generated 387 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:43.462162 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dcb364-370e-4e07-9a1f-b4d078993f46 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:43.482863 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 786/1586] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:15:43 2022] POST /identity/v3/users => generated 290 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:43.484332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48282f06-b674-4b47-8119-065948743dcd admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:43.500906 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48282f06-b674-4b47-8119-065948743dcd admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:43.506810 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:43.557981 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dcb364-370e-4e07-9a1f-b4d078993f46 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:43.563615 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aeb038-bfe0-4bf9-9c12-722daa2e9f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:43.564949 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aeb038-bfe0-4bf9-9c12-722daa2e9f55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:43.565389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aeb038-bfe0-4bf9-9c12-722daa2e9f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:43.565740 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aeb038-bfe0-4bf9-9c12-722daa2e9f55 admin admin] PATH_INFO: `/v3/projects/4a74724882c84e8f893f5be0f59a9581/users/fff9f1fc84a54b83a7a384965f40c9d9/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:43.584508 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aeb038-bfe0-4bf9-9c12-722daa2e9f55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a74724882c84e8f893f5be0f59a9581, user_id=fff9f1fc84a54b83a7a384965f40c9d9, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:43.585669 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: WARNING py.warnings [None req-29aeb038-bfe0-4bf9-9c12-722daa2e9f55 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 21:15:43.585669 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: warnings.warn(msg) Jun 27 21:15:43.585669 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: Jun 27 21:15:43.587859 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aeb038-bfe0-4bf9-9c12-722daa2e9f55 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:43.596443 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e19e64f-65ac-41ae-80fa-4501ae55ff27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:43.597491 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19e64f-65ac-41ae-80fa-4501ae55ff27 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:43.597985 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19e64f-65ac-41ae-80fa-4501ae55ff27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:43.598474 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19e64f-65ac-41ae-80fa-4501ae55ff27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:43.600854 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e19e64f-65ac-41ae-80fa-4501ae55ff27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:43.604535 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e19e64f-65ac-41ae-80fa-4501ae55ff27 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:43.606422 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 801/1587] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:15:43 2022] PUT /identity/v3/projects/4a74724882c84e8f893f5be0f59a9581/users/fff9f1fc84a54b83a7a384965f40c9d9/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:15:43.607408 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aeb038-bfe0-4bf9-9c12-722daa2e9f55 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:43.638944 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 787/1588] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:43.640413 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e19e64f-65ac-41ae-80fa-4501ae55ff27 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:43.662884 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e19e64f-65ac-41ae-80fa-4501ae55ff27 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:43.716124 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aeb038-bfe0-4bf9-9c12-722daa2e9f55 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:43.725954 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6806f24e-4b7b-44af-95e4-4fc0977ee6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:43.727181 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806f24e-4b7b-44af-95e4-4fc0977ee6e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:43.727682 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806f24e-4b7b-44af-95e4-4fc0977ee6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:43.728195 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806f24e-4b7b-44af-95e4-4fc0977ee6e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:43.729811 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6806f24e-4b7b-44af-95e4-4fc0977ee6e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:43.733642 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6806f24e-4b7b-44af-95e4-4fc0977ee6e9 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:43.739771 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:43.766878 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 802/1589] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:43.769103 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6806f24e-4b7b-44af-95e4-4fc0977ee6e9 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:43.791562 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6806f24e-4b7b-44af-95e4-4fc0977ee6e9 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:43.802674 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb52fe6e-bb5c-4920-b216-7f6a70c43a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:43.804467 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52fe6e-bb5c-4920-b216-7f6a70c43a24 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:43.805206 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52fe6e-bb5c-4920-b216-7f6a70c43a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:43.805644 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52fe6e-bb5c-4920-b216-7f6a70c43a24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:43.807339 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb52fe6e-bb5c-4920-b216-7f6a70c43a24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:43.808540 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db7bcc8-eb91-4680-8b06-477b4bf4a091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:43.810122 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db7bcc8-eb91-4680-8b06-477b4bf4a091 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:43.810737 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db7bcc8-eb91-4680-8b06-477b4bf4a091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:43.810944 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb52fe6e-bb5c-4920-b216-7f6a70c43a24 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:43.811266 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db7bcc8-eb91-4680-8b06-477b4bf4a091 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:43.812981 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db7bcc8-eb91-4680-8b06-477b4bf4a091 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:43.817422 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db7bcc8-eb91-4680-8b06-477b4bf4a091 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:43.841076 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 788/1590] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:43.841076 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db7bcc8-eb91-4680-8b06-477b4bf4a091 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:43.842082 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 803/1591] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:43.843805 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb52fe6e-bb5c-4920-b216-7f6a70c43a24 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:43.859195 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db7bcc8-eb91-4680-8b06-477b4bf4a091 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:43.861349 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb52fe6e-bb5c-4920-b216-7f6a70c43a24 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:43.867218 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebfe08c-e36c-48fd-85b1-5f9ddd7a0c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:43.868393 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebfe08c-e36c-48fd-85b1-5f9ddd7a0c10 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:43.869137 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebfe08c-e36c-48fd-85b1-5f9ddd7a0c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:43.869856 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aba1de-e6d3-4d7d-a81e-8fe2b2b970c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:43.870285 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebfe08c-e36c-48fd-85b1-5f9ddd7a0c10 admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:43.871074 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aba1de-e6d3-4d7d-a81e-8fe2b2b970c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:43.871653 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aba1de-e6d3-4d7d-a81e-8fe2b2b970c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:43.871960 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebfe08c-e36c-48fd-85b1-5f9ddd7a0c10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:43.872310 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aba1de-e6d3-4d7d-a81e-8fe2b2b970c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:43.873823 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aba1de-e6d3-4d7d-a81e-8fe2b2b970c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:43.875400 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebfe08c-e36c-48fd-85b1-5f9ddd7a0c10 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:43.877195 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aba1de-e6d3-4d7d-a81e-8fe2b2b970c2 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:43.899110 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 804/1592] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:43.901926 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dcae5a-13a2-4724-8dc5-cc53ab3d78ff None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:43.902793 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dcae5a-13a2-4724-8dc5-cc53ab3d78ff None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:43.903298 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dcae5a-13a2-4724-8dc5-cc53ab3d78ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:43.908211 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 789/1593] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:15:43 2022] POST /identity/v3/users => generated 294 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:43.909938 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebfe08c-e36c-48fd-85b1-5f9ddd7a0c10 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:43.923756 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebfe08c-e36c-48fd-85b1-5f9ddd7a0c10 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:43.929025 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188d1dc4-d391-45fd-9baf-678095e4324b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:43.929811 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d1dc4-d391-45fd-9baf-678095e4324b admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:43.930169 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d1dc4-d391-45fd-9baf-678095e4324b admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:43.930497 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d1dc4-d391-45fd-9baf-678095e4324b admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:43.931302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188d1dc4-d391-45fd-9baf-678095e4324b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:43.933195 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188d1dc4-d391-45fd-9baf-678095e4324b admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:43.936034 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-93dcae5a-13a2-4724-8dc5-cc53ab3d78ff None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:43.938133 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-93dcae5a-13a2-4724-8dc5-cc53ab3d78ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:43.949908 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 790/1594] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:43.949908 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188d1dc4-d391-45fd-9baf-678095e4324b admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:43.954128 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 805/1595] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:15:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:43.954941 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dcae5a-13a2-4724-8dc5-cc53ab3d78ff None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:43.963377 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188d1dc4-d391-45fd-9baf-678095e4324b admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:43.970081 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86007f71-6fc9-4af9-ab48-03ff02fba835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:43.970686 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86007f71-6fc9-4af9-ab48-03ff02fba835 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:43.970971 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86007f71-6fc9-4af9-ab48-03ff02fba835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:43.971244 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86007f71-6fc9-4af9-ab48-03ff02fba835 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:43.971700 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dcae5a-13a2-4724-8dc5-cc53ab3d78ff None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:43.972035 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86007f71-6fc9-4af9-ab48-03ff02fba835 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:43.973701 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86007f71-6fc9-4af9-ab48-03ff02fba835 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:43.979742 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780539ac-07d4-4624-b4b5-f0f5c0d4d21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:43.981097 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780539ac-07d4-4624-b4b5-f0f5c0d4d21d admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:43.981569 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780539ac-07d4-4624-b4b5-f0f5c0d4d21d admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:43.982061 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780539ac-07d4-4624-b4b5-f0f5c0d4d21d admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:43.983497 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780539ac-07d4-4624-b4b5-f0f5c0d4d21d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:43.986354 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 791/1596] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:43.986958 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780539ac-07d4-4624-b4b5-f0f5c0d4d21d admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:43.987085 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86007f71-6fc9-4af9-ab48-03ff02fba835 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:43.998650 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86007f71-6fc9-4af9-ab48-03ff02fba835 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:44.003577 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca48a1-4f2a-4990-b9bb-74766e9f456e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:44.004977 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca48a1-4f2a-4990-b9bb-74766e9f456e admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:44.004977 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca48a1-4f2a-4990-b9bb-74766e9f456e admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:44.005205 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca48a1-4f2a-4990-b9bb-74766e9f456e admin admin] PATH_INFO: `/v3/projects/e1509bd3afb34378b8e8327d4a3af3c1/users/0aea47fd18bf4931a04d34271405f21f/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:44.008285 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 806/1597] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:44.010071 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780539ac-07d4-4624-b4b5-f0f5c0d4d21d admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:44.031622 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780539ac-07d4-4624-b4b5-f0f5c0d4d21d admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:44.036459 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca48a1-4f2a-4990-b9bb-74766e9f456e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1509bd3afb34378b8e8327d4a3af3c1, user_id=0aea47fd18bf4931a04d34271405f21f, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:44.037976 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e55209-4545-4bee-9055-0404956ab046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:44.038836 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e55209-4545-4bee-9055-0404956ab046 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:44.039295 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e55209-4545-4bee-9055-0404956ab046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:44.039648 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e55209-4545-4bee-9055-0404956ab046 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:44.042904 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca48a1-4f2a-4990-b9bb-74766e9f456e admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:44.043790 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e55209-4545-4bee-9055-0404956ab046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:44.045491 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e55209-4545-4bee-9055-0404956ab046 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:44.062205 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 807/1598] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:44 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:44.062205 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e55209-4545-4bee-9055-0404956ab046 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:44.070647 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 792/1599] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:15:43 2022] PUT /identity/v3/projects/e1509bd3afb34378b8e8327d4a3af3c1/users/0aea47fd18bf4931a04d34271405f21f/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:15:44.072119 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca48a1-4f2a-4990-b9bb-74766e9f456e admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:44.078615 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e55209-4545-4bee-9055-0404956ab046 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:44.140454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:44.190046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02be452-2431-4178-963c-0b1e9e21edd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:44.191162 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02be452-2431-4178-963c-0b1e9e21edd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:44.191603 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02be452-2431-4178-963c-0b1e9e21edd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:44.192063 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02be452-2431-4178-963c-0b1e9e21edd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:44.193634 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02be452-2431-4178-963c-0b1e9e21edd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:44.196884 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02be452-2431-4178-963c-0b1e9e21edd7 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:44.210304 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca48a1-4f2a-4990-b9bb-74766e9f456e admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:44.223019 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b88a58-278d-4674-922d-28af2d023273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:44.224367 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b88a58-278d-4674-922d-28af2d023273 admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:44.224773 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b88a58-278d-4674-922d-28af2d023273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:44.225188 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b88a58-278d-4674-922d-28af2d023273 admin admin] PATH_INFO: `/v3/projects/cd3c005f16c6425fbd61a8c40dcf89fd/users/5223bd5ca2f546d191b9b65a8c71724a/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:44.240028 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 808/1600] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:44 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:44.240028 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02be452-2431-4178-963c-0b1e9e21edd7 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:44.258872 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b88a58-278d-4674-922d-28af2d023273 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd3c005f16c6425fbd61a8c40dcf89fd, user_id=5223bd5ca2f546d191b9b65a8c71724a, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:44.265843 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b88a58-278d-4674-922d-28af2d023273 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:44.316589 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 793/1601] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:15:44 2022] PUT /identity/v3/projects/cd3c005f16c6425fbd61a8c40dcf89fd/users/5223bd5ca2f546d191b9b65a8c71724a/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:15:44.316589 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b88a58-278d-4674-922d-28af2d023273 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:44.402874 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02be452-2431-4178-963c-0b1e9e21edd7 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:44.436110 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b88a58-278d-4674-922d-28af2d023273 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:44.450239 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436421e-28cc-47e6-ac4d-77fa2021d5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:44.451695 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436421e-28cc-47e6-ac4d-77fa2021d5a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:44.451695 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436421e-28cc-47e6-ac4d-77fa2021d5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:44.451695 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436421e-28cc-47e6-ac4d-77fa2021d5a4 admin admin] PATH_INFO: `/v3/projects/d8c1bcb797ba4eb0ad6daf5ed39baa89/users/7969c6620a0d406890fb4c15b58ab349/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:44.471724 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9436421e-28cc-47e6-ac4d-77fa2021d5a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8c1bcb797ba4eb0ad6daf5ed39baa89, user_id=7969c6620a0d406890fb4c15b58ab349, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:44.472102 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d06f80-738c-43cd-b99b-ffcbe82ab8dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:44.472811 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d06f80-738c-43cd-b99b-ffcbe82ab8dc service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:44.473190 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d06f80-738c-43cd-b99b-ffcbe82ab8dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:44.473382 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d06f80-738c-43cd-b99b-ffcbe82ab8dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:44.475689 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9436421e-28cc-47e6-ac4d-77fa2021d5a4 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:44.486604 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d06f80-738c-43cd-b99b-ffcbe82ab8dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:44.488509 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d06f80-738c-43cd-b99b-ffcbe82ab8dc service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:44.502799 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 809/1603] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:15:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:15:44.502799 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d06f80-738c-43cd-b99b-ffcbe82ab8dc service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:44.503514 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 794/1603] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:15:44 2022] PUT /identity/v3/projects/d8c1bcb797ba4eb0ad6daf5ed39baa89/users/7969c6620a0d406890fb4c15b58ab349/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:15:44.505348 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436421e-28cc-47e6-ac4d-77fa2021d5a4 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:44.508236 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:44.517060 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d06f80-738c-43cd-b99b-ffcbe82ab8dc service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:44.604673 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436421e-28cc-47e6-ac4d-77fa2021d5a4 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:44.611512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f825173-e879-4f98-a108-6023a05c39fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:44.612554 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f825173-e879-4f98-a108-6023a05c39fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:44.624867 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f825173-e879-4f98-a108-6023a05c39fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:44.625365 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f825173-e879-4f98-a108-6023a05c39fd admin admin] PATH_INFO: `/v3/projects/a2e58fc8b6e14185b4409884a5701316/users/db0e1b60083640bd89b9b4287fdb9d28/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:44.650796 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b0752-0753-48c1-a5f5-26632660a2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:44.653275 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b0752-0753-48c1-a5f5-26632660a2f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:44.653728 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b0752-0753-48c1-a5f5-26632660a2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:44.654109 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b0752-0753-48c1-a5f5-26632660a2f2 admin admin] PATH_INFO: `/v3/projects/ec2655c9c3da4ee1b96b589e336fa5ea/users/123912b9f9a345f5bef7c88529b37838/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:44.670614 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f825173-e879-4f98-a108-6023a05c39fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2e58fc8b6e14185b4409884a5701316, user_id=db0e1b60083640bd89b9b4287fdb9d28, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:44.677251 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f825173-e879-4f98-a108-6023a05c39fd admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:44.686048 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8b0752-0753-48c1-a5f5-26632660a2f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec2655c9c3da4ee1b96b589e336fa5ea, user_id=123912b9f9a345f5bef7c88529b37838, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:44.690931 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8b0752-0753-48c1-a5f5-26632660a2f2 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:44.708481 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 795/1604] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:15:44 2022] PUT /identity/v3/projects/a2e58fc8b6e14185b4409884a5701316/users/db0e1b60083640bd89b9b4287fdb9d28/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:15:44.710309 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26cb8e-41d7-42ad-9d1f-380278135bb5 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:44.710606 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26cb8e-41d7-42ad-9d1f-380278135bb5 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:44.711071 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26cb8e-41d7-42ad-9d1f-380278135bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:44.727120 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 810/1605] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:15:44 2022] PUT /identity/v3/projects/ec2655c9c3da4ee1b96b589e336fa5ea/users/123912b9f9a345f5bef7c88529b37838/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:15:44.730552 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b0752-0753-48c1-a5f5-26632660a2f2 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:44.747160 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-ee26cb8e-41d7-42ad-9d1f-380278135bb5 None None] MFA Rules not processed for user `0aea47fd18bf4931a04d34271405f21f`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:44.748509 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-ee26cb8e-41d7-42ad-9d1f-380278135bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:44.767922 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:44.850635 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 796/1606] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:15:44 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:44.856108 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fadd5a-6732-4fa2-b1fb-63b1aaf5b7ea None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:44.856417 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fadd5a-6732-4fa2-b1fb-63b1aaf5b7ea None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:44.856772 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fadd5a-6732-4fa2-b1fb-63b1aaf5b7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:44.872187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b0752-0753-48c1-a5f5-26632660a2f2 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:44.878904 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5059a2d5-9720-4d34-8cb4-66180e602154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:44.879841 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059a2d5-9720-4d34-8cb4-66180e602154 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:44.880217 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059a2d5-9720-4d34-8cb4-66180e602154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:44.880585 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059a2d5-9720-4d34-8cb4-66180e602154 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:44.881486 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5059a2d5-9720-4d34-8cb4-66180e602154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:44.883516 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5059a2d5-9720-4d34-8cb4-66180e602154 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:44.891511 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-55fadd5a-6732-4fa2-b1fb-63b1aaf5b7ea None None] MFA Rules not processed for user `5223bd5ca2f546d191b9b65a8c71724a`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:44.892835 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-55fadd5a-6732-4fa2-b1fb-63b1aaf5b7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:44.897165 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 811/1607] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:44 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:44.898671 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32e0dd-ab3b-40eb-9306-9acd31685f22 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:44.899060 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32e0dd-ab3b-40eb-9306-9acd31685f22 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:44.899444 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32e0dd-ab3b-40eb-9306-9acd31685f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:44.921729 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-dc32e0dd-ab3b-40eb-9306-9acd31685f22 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:44.923118 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-dc32e0dd-ab3b-40eb-9306-9acd31685f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:44.934815 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 812/1608] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:15:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:44.946961 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958f370-f47a-4776-a546-98110e8416d6 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:44.965257 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958f370-f47a-4776-a546-98110e8416d6 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:44.967777 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958f370-f47a-4776-a546-98110e8416d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:44.988501 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 797/1609] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:15:44 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:44.988501 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb22571-08cd-4201-9fdd-fc6e49e3e642 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:44.988501 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb22571-08cd-4201-9fdd-fc6e49e3e642 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:44.988501 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb22571-08cd-4201-9fdd-fc6e49e3e642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:44.998549 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-6958f370-f47a-4776-a546-98110e8416d6 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:45.000765 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-6958f370-f47a-4776-a546-98110e8416d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:45.013500 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 813/1610] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:15:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:45.014375 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958f370-f47a-4776-a546-98110e8416d6 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:45.032870 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-9eb22571-08cd-4201-9fdd-fc6e49e3e642 None None] MFA Rules not processed for user `123912b9f9a345f5bef7c88529b37838`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:45.033761 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-9eb22571-08cd-4201-9fdd-fc6e49e3e642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:45.042320 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958f370-f47a-4776-a546-98110e8416d6 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:45.054973 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f09fa2-6df1-4cb2-8e7f-4bc00740e71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:45.056695 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f09fa2-6df1-4cb2-8e7f-4bc00740e71e admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:45.057832 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f09fa2-6df1-4cb2-8e7f-4bc00740e71e admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:45.059165 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f09fa2-6df1-4cb2-8e7f-4bc00740e71e admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:45.060996 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f09fa2-6df1-4cb2-8e7f-4bc00740e71e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:45.064112 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f09fa2-6df1-4cb2-8e7f-4bc00740e71e admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:45.109132 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 814/1611] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:15:45 2022] POST /identity/v3/projects => generated 363 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:45.110368 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f09fa2-6df1-4cb2-8e7f-4bc00740e71e admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:45.160878 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 798/1612] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:15:44 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:45.161888 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb22571-08cd-4201-9fdd-fc6e49e3e642 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:45.226164 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:45.257567 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:45.274882 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f09fa2-6df1-4cb2-8e7f-4bc00740e71e admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:45.285549 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92bcd5-5bb5-4d83-bb90-15aa38e0a2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:45.287722 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92bcd5-5bb5-4d83-bb90-15aa38e0a2d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:45.288003 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92bcd5-5bb5-4d83-bb90-15aa38e0a2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:45.288267 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a92bcd5-5bb5-4d83-bb90-15aa38e0a2d5 admin admin] PATH_INFO: `/v3/projects/0cdaf724b92644a0ae7fbb71147aef40/users/ba0127dd69da407ab096773fc9e22530/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:45.297592 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb22571-08cd-4201-9fdd-fc6e49e3e642 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:45.334667 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a92bcd5-5bb5-4d83-bb90-15aa38e0a2d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cdaf724b92644a0ae7fbb71147aef40, user_id=ba0127dd69da407ab096773fc9e22530, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:45.346613 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a92bcd5-5bb5-4d83-bb90-15aa38e0a2d5 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:45.405337 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 815/1613] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:15:45 2022] PUT /identity/v3/projects/0cdaf724b92644a0ae7fbb71147aef40/users/ba0127dd69da407ab096773fc9e22530/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:15:45.407930 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92bcd5-5bb5-4d83-bb90-15aa38e0a2d5 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:45.444053 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166674dc-ae84-4591-b63e-2a8e70015337 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:45.445050 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166674dc-ae84-4591-b63e-2a8e70015337 service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:45.445436 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166674dc-ae84-4591-b63e-2a8e70015337 service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:45.445810 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166674dc-ae84-4591-b63e-2a8e70015337 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:45.459929 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166674dc-ae84-4591-b63e-2a8e70015337 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:45.461786 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166674dc-ae84-4591-b63e-2a8e70015337 service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:45.474989 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 799/1614] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:15:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:15:45.476015 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166674dc-ae84-4591-b63e-2a8e70015337 service neutron] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:45.486460 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166674dc-ae84-4591-b63e-2a8e70015337 service neutron] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:45.560658 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba4a27a-43b5-4698-9eb4-e692af384f81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:45.562603 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4a27a-43b5-4698-9eb4-e692af384f81 service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:45.563160 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4a27a-43b5-4698-9eb4-e692af384f81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:45.563664 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4a27a-43b5-4698-9eb4-e692af384f81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:45.654299 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a92bcd5-5bb5-4d83-bb90-15aa38e0a2d5 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:45.678086 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ea5a5-4f9f-4207-a206-a24c530efa41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:45.679530 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ea5a5-4f9f-4207-a206-a24c530efa41 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:45.680039 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ea5a5-4f9f-4207-a206-a24c530efa41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:45.680587 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ea5a5-4f9f-4207-a206-a24c530efa41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:45.682326 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995ea5a5-4f9f-4207-a206-a24c530efa41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:45.687240 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995ea5a5-4f9f-4207-a206-a24c530efa41 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:45.706971 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba4a27a-43b5-4698-9eb4-e692af384f81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:45.709044 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba4a27a-43b5-4698-9eb4-e692af384f81 service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:45.722851 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 800/1615] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:15:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:15:45.724153 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba4a27a-43b5-4698-9eb4-e692af384f81 service neutron] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:45.736733 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba4a27a-43b5-4698-9eb4-e692af384f81 service neutron] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:45.748191 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b9dba-c3ff-4f61-b117-0384063a6d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:45.749551 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b9dba-c3ff-4f61-b117-0384063a6d12 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:45.750076 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b9dba-c3ff-4f61-b117-0384063a6d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:45.751363 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b9dba-c3ff-4f61-b117-0384063a6d12 admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:45.752559 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b9dba-c3ff-4f61-b117-0384063a6d12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:45.756371 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b9dba-c3ff-4f61-b117-0384063a6d12 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:45.794226 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 801/1616] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:15:45 2022] POST /identity/v3/users => generated 288 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:45.795331 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b9dba-c3ff-4f61-b117-0384063a6d12 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:45.807202 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 816/1617] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:15:45 2022] POST /identity/v3/projects => generated 357 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:45.812140 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da882740-2fc3-4e07-a4db-c332c3da3167 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:45.813912 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da882740-2fc3-4e07-a4db-c332c3da3167 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:45.815067 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da882740-2fc3-4e07-a4db-c332c3da3167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:45.841236 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:45.877948 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-da882740-2fc3-4e07-a4db-c332c3da3167 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:45.881322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-da882740-2fc3-4e07-a4db-c332c3da3167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:45.884687 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b9dba-c3ff-4f61-b117-0384063a6d12 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:46.030849 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a0cc54-deba-4658-ae44-6f6aeb5eb386 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:46.031562 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a0cc54-deba-4658-ae44-6f6aeb5eb386 service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:46.031846 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a0cc54-deba-4658-ae44-6f6aeb5eb386 service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:46.032122 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a0cc54-deba-4658-ae44-6f6aeb5eb386 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:46.224810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 817/1618] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:15:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:46.224810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da882740-2fc3-4e07-a4db-c332c3da3167 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:46.224810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da882740-2fc3-4e07-a4db-c332c3da3167 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:46.224810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26524f6f-b469-489f-a0a2-e4ad18e85372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:46.226332 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26524f6f-b469-489f-a0a2-e4ad18e85372 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:46.227523 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a0cc54-deba-4658-ae44-6f6aeb5eb386 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:46.227676 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26524f6f-b469-489f-a0a2-e4ad18e85372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:46.227676 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26524f6f-b469-489f-a0a2-e4ad18e85372 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:46.227676 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26524f6f-b469-489f-a0a2-e4ad18e85372 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:46.227945 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a0cc54-deba-4658-ae44-6f6aeb5eb386 service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:46.228031 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26524f6f-b469-489f-a0a2-e4ad18e85372 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:46.228113 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 802/1619] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:15:45 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:15:46.228194 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 818/1620] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:46.228278 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a0cc54-deba-4658-ae44-6f6aeb5eb386 service cinder] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:46.228365 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26524f6f-b469-489f-a0a2-e4ad18e85372 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:46.242225 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a0cc54-deba-4658-ae44-6f6aeb5eb386 service cinder] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:46.255534 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39989d8b-fd77-49d9-aaf9-0bcde8da5c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:46.256314 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39989d8b-fd77-49d9-aaf9-0bcde8da5c1f admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:46.256603 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39989d8b-fd77-49d9-aaf9-0bcde8da5c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:46.257089 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39989d8b-fd77-49d9-aaf9-0bcde8da5c1f admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:46.258016 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39989d8b-fd77-49d9-aaf9-0bcde8da5c1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:46.264019 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39989d8b-fd77-49d9-aaf9-0bcde8da5c1f admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:46.312375 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 803/1621] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:15:46 2022] POST /identity/v3/users => generated 285 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:46.312375 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39989d8b-fd77-49d9-aaf9-0bcde8da5c1f admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:46.344568 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39989d8b-fd77-49d9-aaf9-0bcde8da5c1f admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:46.348434 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:46.357125 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6153d606-fe75-401b-a2ce-219f62a343cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:46.359854 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6153d606-fe75-401b-a2ce-219f62a343cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:46.360233 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6153d606-fe75-401b-a2ce-219f62a343cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:46.360524 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6153d606-fe75-401b-a2ce-219f62a343cc admin admin] PATH_INFO: `/v3/projects/3a75f088b63d48eb94701792206e1a8d/users/9cbe6796c5d04056b1f2104c65ce27d5/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:46.375091 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26524f6f-b469-489f-a0a2-e4ad18e85372 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:46.384888 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66f032-2577-4a82-9f37-921b64679381 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:46.386567 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66f032-2577-4a82-9f37-921b64679381 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:46.388117 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66f032-2577-4a82-9f37-921b64679381 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:46.388651 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66f032-2577-4a82-9f37-921b64679381 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:46.405711 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6153d606-fe75-401b-a2ce-219f62a343cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a75f088b63d48eb94701792206e1a8d, user_id=9cbe6796c5d04056b1f2104c65ce27d5, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:46.413134 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6153d606-fe75-401b-a2ce-219f62a343cc admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:46.415015 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b66f032-2577-4a82-9f37-921b64679381 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:46.420984 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b66f032-2577-4a82-9f37-921b64679381 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:46.448029 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 804/1622] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:15:46 2022] PUT /identity/v3/projects/3a75f088b63d48eb94701792206e1a8d/users/9cbe6796c5d04056b1f2104c65ce27d5/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:15:46.453451 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6153d606-fe75-401b-a2ce-219f62a343cc admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:46.455966 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 819/1623] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:15:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:15:46.459404 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66f032-2577-4a82-9f37-921b64679381 service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:46.511770 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66f032-2577-4a82-9f37-921b64679381 service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:46.622368 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:46.638323 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6153d606-fe75-401b-a2ce-219f62a343cc admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:46.652985 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a1656-93a2-4dc6-81bc-bdc71eb59e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:46.653701 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a1656-93a2-4dc6-81bc-bdc71eb59e97 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:46.653987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a1656-93a2-4dc6-81bc-bdc71eb59e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:46.654265 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a1656-93a2-4dc6-81bc-bdc71eb59e97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:46.655028 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320a1656-93a2-4dc6-81bc-bdc71eb59e97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:46.656697 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320a1656-93a2-4dc6-81bc-bdc71eb59e97 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:46.687286 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 805/1624] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:46.689337 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a1656-93a2-4dc6-81bc-bdc71eb59e97 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:46.712695 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a1656-93a2-4dc6-81bc-bdc71eb59e97 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:46.725951 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54a2e8-e0a7-427e-b4c2-1577e7ea726f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:46.726467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4a553-ce28-4a75-918b-95e5e671e9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:46.727454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4a553-ce28-4a75-918b-95e5e671e9df admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:46.727745 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4a553-ce28-4a75-918b-95e5e671e9df admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:46.728025 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4a553-ce28-4a75-918b-95e5e671e9df admin admin] PATH_INFO: `/v3/projects/ceed20f4961c424f8e24835a581b46ac/users/b1048172079c493cabce8b27e86c5a68/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:46.729098 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54a2e8-e0a7-427e-b4c2-1577e7ea726f admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:46.731065 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54a2e8-e0a7-427e-b4c2-1577e7ea726f admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:46.731710 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54a2e8-e0a7-427e-b4c2-1577e7ea726f admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:46.733615 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd54a2e8-e0a7-427e-b4c2-1577e7ea726f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:46.739933 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd54a2e8-e0a7-427e-b4c2-1577e7ea726f admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:46.764602 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 820/1625] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:46.768720 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54a2e8-e0a7-427e-b4c2-1577e7ea726f admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:46.790650 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54a2e8-e0a7-427e-b4c2-1577e7ea726f admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:46.803057 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4a553-ce28-4a75-918b-95e5e671e9df admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceed20f4961c424f8e24835a581b46ac, user_id=b1048172079c493cabce8b27e86c5a68, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:46.803355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3390e4fd-6f46-432e-9c87-b59042321728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:46.805604 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3390e4fd-6f46-432e-9c87-b59042321728 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:46.805959 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3390e4fd-6f46-432e-9c87-b59042321728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:46.806138 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3390e4fd-6f46-432e-9c87-b59042321728 admin admin] PATH_INFO: `/v3/projects/3a75f088b63d48eb94701792206e1a8d/users/9cbe6796c5d04056b1f2104c65ce27d5/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:46.812766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4a553-ce28-4a75-918b-95e5e671e9df admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:46.813431 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3390e4fd-6f46-432e-9c87-b59042321728 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a75f088b63d48eb94701792206e1a8d, user_id=9cbe6796c5d04056b1f2104c65ce27d5, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:46.816461 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3390e4fd-6f46-432e-9c87-b59042321728 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:46.847402 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 821/1626] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:15:46 2022] PUT /identity/v3/projects/3a75f088b63d48eb94701792206e1a8d/users/9cbe6796c5d04056b1f2104c65ce27d5/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:15:46.860353 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b504f73-19ae-4947-aa0b-62c8b9f560d1 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:46.860706 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b504f73-19ae-4947-aa0b-62c8b9f560d1 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:46.861101 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b504f73-19ae-4947-aa0b-62c8b9f560d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:46.868014 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 806/1627] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:15:46 2022] PUT /identity/v3/projects/ceed20f4961c424f8e24835a581b46ac/users/b1048172079c493cabce8b27e86c5a68/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:15:46.873483 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4a553-ce28-4a75-918b-95e5e671e9df admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:46.898572 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-4b504f73-19ae-4947-aa0b-62c8b9f560d1 None None] MFA Rules not processed for user `9cbe6796c5d04056b1f2104c65ce27d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:46.900603 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-4b504f73-19ae-4947-aa0b-62c8b9f560d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:47.073704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4a553-ce28-4a75-918b-95e5e671e9df admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:47.101340 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b1b70-9bff-45f5-83d8-b7b8929ff7b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:47.106015 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b1b70-9bff-45f5-83d8-b7b8929ff7b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:47.107982 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b1b70-9bff-45f5-83d8-b7b8929ff7b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:47.114236 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b1b70-9bff-45f5-83d8-b7b8929ff7b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:47.115895 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446b1b70-9bff-45f5-83d8-b7b8929ff7b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:47.122298 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446b1b70-9bff-45f5-83d8-b7b8929ff7b8 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:47.174855 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 807/1628] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:47.181267 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 822/1629] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:15:46 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:47.185473 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b1b70-9bff-45f5-83d8-b7b8929ff7b8 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:47.201405 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b504f73-19ae-4947-aa0b-62c8b9f560d1 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:47.233878 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b1b70-9bff-45f5-83d8-b7b8929ff7b8 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:47.258133 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87607bf6-d409-4329-97a8-0719fb4513dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:47.259489 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87607bf6-d409-4329-97a8-0719fb4513dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:47.262224 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87607bf6-d409-4329-97a8-0719fb4513dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:47.262851 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87607bf6-d409-4329-97a8-0719fb4513dc admin admin] PATH_INFO: `/v3/projects/ceed20f4961c424f8e24835a581b46ac/users/b1048172079c493cabce8b27e86c5a68/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:47.269509 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87607bf6-d409-4329-97a8-0719fb4513dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceed20f4961c424f8e24835a581b46ac, user_id=b1048172079c493cabce8b27e86c5a68, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:47.273414 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87607bf6-d409-4329-97a8-0719fb4513dc admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:47.302350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 808/1630] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:15:47 2022] PUT /identity/v3/projects/ceed20f4961c424f8e24835a581b46ac/users/b1048172079c493cabce8b27e86c5a68/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:15:47.314429 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e26ebe8-bcda-47b7-a074-8501f8ea8677 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:47.315072 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e26ebe8-bcda-47b7-a074-8501f8ea8677 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:47.315524 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e26ebe8-bcda-47b7-a074-8501f8ea8677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:47.371751 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b504f73-19ae-4947-aa0b-62c8b9f560d1 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:47.381109 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-2e26ebe8-bcda-47b7-a074-8501f8ea8677 None None] MFA Rules not processed for user `b1048172079c493cabce8b27e86c5a68`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:47.382642 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-2e26ebe8-bcda-47b7-a074-8501f8ea8677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:47.394287 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:47.514309 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e8ed9-dd4a-440e-a1d7-75782211c0b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:47.516308 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e8ed9-dd4a-440e-a1d7-75782211c0b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:47.517689 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e8ed9-dd4a-440e-a1d7-75782211c0b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:47.518137 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e8ed9-dd4a-440e-a1d7-75782211c0b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:47.616605 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 809/1631] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:15:47 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:15:47.631150 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e26ebe8-bcda-47b7-a074-8501f8ea8677 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:47.647814 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e26ebe8-bcda-47b7-a074-8501f8ea8677 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:47.656891 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef394eca-9b08-4625-9610-13d9757d1731 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:47.660195 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef394eca-9b08-4625-9610-13d9757d1731 service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:47.661622 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef394eca-9b08-4625-9610-13d9757d1731 service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:47.661941 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef394eca-9b08-4625-9610-13d9757d1731 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:47.694057 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef394eca-9b08-4625-9610-13d9757d1731 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:47.698052 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef394eca-9b08-4625-9610-13d9757d1731 service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:47.720379 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 810/1632] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:15:47 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:15:47.730783 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48e8ed9-dd4a-440e-a1d7-75782211c0b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:47.732450 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48e8ed9-dd4a-440e-a1d7-75782211c0b4 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:47.747518 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 823/1633] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:15:47 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:15:47.987831 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef394eca-9b08-4625-9610-13d9757d1731 service cinder] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:48.005576 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef394eca-9b08-4625-9610-13d9757d1731 service cinder] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:48.024811 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e8ed9-dd4a-440e-a1d7-75782211c0b4 service cinder] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:48.035558 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e8ed9-dd4a-440e-a1d7-75782211c0b4 service cinder] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:48.036574 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbab086-4c41-48c1-8bd1-505b33cac010 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:48.037466 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbab086-4c41-48c1-8bd1-505b33cac010 service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:48.038822 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbab086-4c41-48c1-8bd1-505b33cac010 service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:48.039278 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbab086-4c41-48c1-8bd1-505b33cac010 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:48.054908 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6a14a-8feb-4079-9d6e-783995e415d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:48.055874 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6a14a-8feb-4079-9d6e-783995e415d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:48.059470 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6a14a-8feb-4079-9d6e-783995e415d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:48.060219 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6a14a-8feb-4079-9d6e-783995e415d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:48.110328 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:48.234975 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbab086-4c41-48c1-8bd1-505b33cac010 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:48.237494 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbab086-4c41-48c1-8bd1-505b33cac010 service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:48.254892 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb6a14a-8feb-4079-9d6e-783995e415d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:48.257534 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb6a14a-8feb-4079-9d6e-783995e415d6 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:48.287335 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 811/1634] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:15:47 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:15:48.313473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 824/1635] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:15:48 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:15:50.369056 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7a41a-4628-40c1-9092-dcbcc1cddc41 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:50.370196 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7a41a-4628-40c1-9092-dcbcc1cddc41 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:50.370866 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7a41a-4628-40c1-9092-dcbcc1cddc41 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:50.373292 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 812/1636] 10.222.0.13 () {54 vars in 961 bytes} [Mon Jun 27 21:15:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:15:50.398146 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021140f-f584-44df-ac01-8c1ad044baf2 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:50.399071 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021140f-f584-44df-ac01-8c1ad044baf2 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:50.400148 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021140f-f584-44df-ac01-8c1ad044baf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:50.483309 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-f021140f-f584-44df-ac01-8c1ad044baf2 None None] MFA Rules not processed for user `1366958034d540b4ba2feb421a6afabc`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:50.485827 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-f021140f-f584-44df-ac01-8c1ad044baf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:50.658572 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:50.707679 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 825/1637] 10.222.0.13 () {58 vars in 1058 bytes} [Mon Jun 27 21:15:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:50.714890 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d76b8-c557-4faf-b797-df725f36d7f5 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:50.715388 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d76b8-c557-4faf-b797-df725f36d7f5 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:50.715814 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d76b8-c557-4faf-b797-df725f36d7f5 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:50.717452 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 813/1638] 10.222.0.13 () {54 vars in 912 bytes} [Mon Jun 27 21:15:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:15:50.723545 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb2b68-50b1-4c46-8b12-a3ee3825bc1e None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:50.724029 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb2b68-50b1-4c46-8b12-a3ee3825bc1e None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:50.724567 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb2b68-50b1-4c46-8b12-a3ee3825bc1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:50.762615 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-20eb2b68-50b1-4c46-8b12-a3ee3825bc1e None None] MFA Rules not processed for user `1366958034d540b4ba2feb421a6afabc`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:50.764095 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-20eb2b68-50b1-4c46-8b12-a3ee3825bc1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:50.792985 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 826/1639] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Jun 27 21:15:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:50.808760 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d76b8-c557-4faf-b797-df725f36d7f5 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:50.832603 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d76b8-c557-4faf-b797-df725f36d7f5 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:50.857747 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dfd82b-06c4-4f7c-860e-60a491338eb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:50.858441 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dfd82b-06c4-4f7c-860e-60a491338eb6 service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:50.858725 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dfd82b-06c4-4f7c-860e-60a491338eb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:50.859658 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dfd82b-06c4-4f7c-860e-60a491338eb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:50.902238 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dfd82b-06c4-4f7c-860e-60a491338eb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:50.906151 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dfd82b-06c4-4f7c-860e-60a491338eb6 service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:50.945986 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 814/1640] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:15:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:15:52.591550 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb2b68-50b1-4c46-8b12-a3ee3825bc1e None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:52.949179 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb2b68-50b1-4c46-8b12-a3ee3825bc1e None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:52.965462 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8026e015-83a4-48d4-8dbc-4a62d0d40ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:52.966717 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026e015-83a4-48d4-8dbc-4a62d0d40ef6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:52.967164 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026e015-83a4-48d4-8dbc-4a62d0d40ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:52.967627 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026e015-83a4-48d4-8dbc-4a62d0d40ef6 admin admin] PATH_INFO: `/v3/users/0aea47fd18bf4931a04d34271405f21f` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:52.969516 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8026e015-83a4-48d4-8dbc-4a62d0d40ef6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0aea47fd18bf4931a04d34271405f21f)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:52.971321 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: WARNING py.warnings [None req-8026e015-83a4-48d4-8dbc-4a62d0d40ef6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 21:15:52.971321 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: warnings.warn(msg) Jun 27 21:15:52.971321 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: Jun 27 21:15:52.985679 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8026e015-83a4-48d4-8dbc-4a62d0d40ef6 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:53.302598 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8026e015-83a4-48d4-8dbc-4a62d0d40ef6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0aea47fd18bf4931a04d34271405f21f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:53.368341 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8026e015-83a4-48d4-8dbc-4a62d0d40ef6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0aea47fd18bf4931a04d34271405f21f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:53.373535 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8026e015-83a4-48d4-8dbc-4a62d0d40ef6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0aea47fd18bf4931a04d34271405f21f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:53.374374 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8026e015-83a4-48d4-8dbc-4a62d0d40ef6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0aea47fd18bf4931a04d34271405f21f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:53.392251 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8026e015-83a4-48d4-8dbc-4a62d0d40ef6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0aea47fd18bf4931a04d34271405f21f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:53.573742 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 827/1641] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:15:52 2022] DELETE /identity/v3/users/0aea47fd18bf4931a04d34271405f21f => generated 0 bytes in 983 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:15:53.578301 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf91c6e-05cb-4176-a208-de3007aaa7b8 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:53.578896 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf91c6e-05cb-4176-a208-de3007aaa7b8 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:53.579078 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf91c6e-05cb-4176-a208-de3007aaa7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:53.650522 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-7bf91c6e-05cb-4176-a208-de3007aaa7b8 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:53.653014 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-7bf91c6e-05cb-4176-a208-de3007aaa7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:53.982585 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 815/1642] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:15:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 405 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:54.059811 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8026e015-83a4-48d4-8dbc-4a62d0d40ef6 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:54.062416 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-8026e015-83a4-48d4-8dbc-4a62d0d40ef6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:54.279736 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1cdbb-7024-4fec-94ed-b5cf589e242b None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:54.280021 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1cdbb-7024-4fec-94ed-b5cf589e242b None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:54.280578 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1cdbb-7024-4fec-94ed-b5cf589e242b None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:54.282881 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 816/1643] 10.222.0.13 () {54 vars in 961 bytes} [Mon Jun 27 21:15:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:15:54.294948 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da573c42-0814-4f1c-830a-527b470840a1 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:54.295327 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da573c42-0814-4f1c-830a-527b470840a1 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:54.295503 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da573c42-0814-4f1c-830a-527b470840a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:54.307418 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8026e015-83a4-48d4-8dbc-4a62d0d40ef6 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:54.334385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930108d7-51e4-4a7f-82a6-6719cd183d5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:54.335093 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930108d7-51e4-4a7f-82a6-6719cd183d5a service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:54.335370 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930108d7-51e4-4a7f-82a6-6719cd183d5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:54.335641 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930108d7-51e4-4a7f-82a6-6719cd183d5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:54.346201 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-da573c42-0814-4f1c-830a-527b470840a1 None None] MFA Rules not processed for user `1366958034d540b4ba2feb421a6afabc`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:54.348380 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-da573c42-0814-4f1c-830a-527b470840a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:54.396085 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930108d7-51e4-4a7f-82a6-6719cd183d5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:54.397666 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930108d7-51e4-4a7f-82a6-6719cd183d5a service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:54.438347 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 828/1644] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:15:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:15:54.576189 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 817/1645] 10.222.0.13 () {58 vars in 1058 bytes} [Mon Jun 27 21:15:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:15:54.583102 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e1794-a350-4719-96be-cd5f100c37de None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:54.584284 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e1794-a350-4719-96be-cd5f100c37de None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:54.584284 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e1794-a350-4719-96be-cd5f100c37de None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:54.585426 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 829/1646] 10.222.0.13 () {54 vars in 912 bytes} [Mon Jun 27 21:15:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:15:54.591967 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defae879-0efb-48f7-bcb4-f3776c257a7a None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:54.592826 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defae879-0efb-48f7-bcb4-f3776c257a7a None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:54.592826 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defae879-0efb-48f7-bcb4-f3776c257a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:54.638026 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-defae879-0efb-48f7-bcb4-f3776c257a7a None None] MFA Rules not processed for user `1366958034d540b4ba2feb421a6afabc`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:54.639311 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-defae879-0efb-48f7-bcb4-f3776c257a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:54.681046 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 818/1647] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Jun 27 21:15:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:15:54.702130 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e1794-a350-4719-96be-cd5f100c37de None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:54.705169 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-fa1e1794-a350-4719-96be-cd5f100c37de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:54.733396 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:54.878830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e1794-a350-4719-96be-cd5f100c37de None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:54.887817 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b7d27-750a-4a99-932a-1dee5591bacb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:54.889299 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b7d27-750a-4a99-932a-1dee5591bacb service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:54.889723 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b7d27-750a-4a99-932a-1dee5591bacb service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:54.890183 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b7d27-750a-4a99-932a-1dee5591bacb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:54.928684 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05b7d27-750a-4a99-932a-1dee5591bacb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:54.937430 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05b7d27-750a-4a99-932a-1dee5591bacb service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:54.976480 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 830/1648] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:15:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:15:55.899038 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defae879-0efb-48f7-bcb4-f3776c257a7a None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:55.901648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-defae879-0efb-48f7-bcb4-f3776c257a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:55.941961 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defae879-0efb-48f7-bcb4-f3776c257a7a None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:55.957270 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03258c14-c60e-4f45-9aa0-9c341a7dff99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:55.958227 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03258c14-c60e-4f45-9aa0-9c341a7dff99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:55.958514 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03258c14-c60e-4f45-9aa0-9c341a7dff99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:55.958807 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03258c14-c60e-4f45-9aa0-9c341a7dff99 admin admin] PATH_INFO: `/v3/projects/e1509bd3afb34378b8e8327d4a3af3c1` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:55.969435 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03258c14-c60e-4f45-9aa0-9c341a7dff99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1509bd3afb34378b8e8327d4a3af3c1)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:55.970504 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: WARNING py.warnings [None req-03258c14-c60e-4f45-9aa0-9c341a7dff99 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 27 21:15:55.970504 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: warnings.warn(msg) Jun 27 21:15:55.970504 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: Jun 27 21:15:55.971898 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03258c14-c60e-4f45-9aa0-9c341a7dff99 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:56.066652 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:56.133317 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-03258c14-c60e-4f45-9aa0-9c341a7dff99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1509bd3afb34378b8e8327d4a3af3c1', 'request_id': 'req-03258c14-c60e-4f45-9aa0-9c341a7dff99'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:56.165419 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-03258c14-c60e-4f45-9aa0-9c341a7dff99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1509bd3afb34378b8e8327d4a3af3c1', 'request_id': 'req-03258c14-c60e-4f45-9aa0-9c341a7dff99'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:56.186720 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-03258c14-c60e-4f45-9aa0-9c341a7dff99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1509bd3afb34378b8e8327d4a3af3c1', 'request_id': 'req-03258c14-c60e-4f45-9aa0-9c341a7dff99'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:56.209315 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-03258c14-c60e-4f45-9aa0-9c341a7dff99 admin admin] The token cache is being invalidate because project e1509bd3afb34378b8e8327d4a3af3c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:15:56.210016 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-03258c14-c60e-4f45-9aa0-9c341a7dff99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:56.211713 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-03258c14-c60e-4f45-9aa0-9c341a7dff99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:56.215086 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 819/1649] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:15:55 2022] DELETE /identity/v3/projects/e1509bd3afb34378b8e8327d4a3af3c1 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:15:56.220301 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b7d27-750a-4a99-932a-1dee5591bacb service cinder] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:56.236062 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b7d27-750a-4a99-932a-1dee5591bacb service cinder] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:56.313144 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9725a-67db-4054-bb20-32a57b892b9b None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:56.313674 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9725a-67db-4054-bb20-32a57b892b9b None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:56.314238 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9725a-67db-4054-bb20-32a57b892b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:56.387306 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194cdeac-bda6-4e49-8075-adc6e3f12990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:56.390367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194cdeac-bda6-4e49-8075-adc6e3f12990 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:56.390903 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194cdeac-bda6-4e49-8075-adc6e3f12990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:56.391675 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194cdeac-bda6-4e49-8075-adc6e3f12990 admin admin] PATH_INFO: `/v3/users/9cbe6796c5d04056b1f2104c65ce27d5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:56.394761 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194cdeac-bda6-4e49-8075-adc6e3f12990 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cbe6796c5d04056b1f2104c65ce27d5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:56.399497 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194cdeac-bda6-4e49-8075-adc6e3f12990 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:56.402435 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-92c9725a-67db-4054-bb20-32a57b892b9b None None] MFA Rules not processed for user `a0408ee5961249d1bd296a6ef1a030a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:56.407099 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-92c9725a-67db-4054-bb20-32a57b892b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:56.518326 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-194cdeac-bda6-4e49-8075-adc6e3f12990 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cbe6796c5d04056b1f2104c65ce27d5'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:56.537441 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-194cdeac-bda6-4e49-8075-adc6e3f12990 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cbe6796c5d04056b1f2104c65ce27d5'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:56.539927 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-194cdeac-bda6-4e49-8075-adc6e3f12990 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cbe6796c5d04056b1f2104c65ce27d5'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:56.541074 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-194cdeac-bda6-4e49-8075-adc6e3f12990 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cbe6796c5d04056b1f2104c65ce27d5'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:56.558773 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-194cdeac-bda6-4e49-8075-adc6e3f12990 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cbe6796c5d04056b1f2104c65ce27d5'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:56.594400 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 831/1650] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:15:56 2022] DELETE /identity/v3/users/9cbe6796c5d04056b1f2104c65ce27d5 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:15:56.624307 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 820/1651] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:15:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:56.631099 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fcd391-8249-448c-b2a9-71c5f218824f None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:56.631680 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fcd391-8249-448c-b2a9-71c5f218824f None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:56.632162 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fcd391-8249-448c-b2a9-71c5f218824f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:56.664741 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-36fcd391-8249-448c-b2a9-71c5f218824f None None] MFA Rules not processed for user `a0408ee5961249d1bd296a6ef1a030a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:56.669454 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-36fcd391-8249-448c-b2a9-71c5f218824f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:56.730562 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:56.831084 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9725a-67db-4054-bb20-32a57b892b9b None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:56.834817 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-92c9725a-67db-4054-bb20-32a57b892b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:56.836471 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 832/1652] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:15:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:15:57.001897 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9725a-67db-4054-bb20-32a57b892b9b None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:57.128672 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:57.212985 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18538ae2-fba5-4177-8ac2-f657cd2cd71b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:57.219766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18538ae2-fba5-4177-8ac2-f657cd2cd71b service nova] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:57.221488 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18538ae2-fba5-4177-8ac2-f657cd2cd71b service nova] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:57.221940 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18538ae2-fba5-4177-8ac2-f657cd2cd71b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:57.250572 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18538ae2-fba5-4177-8ac2-f657cd2cd71b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:57.252428 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18538ae2-fba5-4177-8ac2-f657cd2cd71b service nova] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:57.297430 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 821/1653] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:15:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:15:57.423882 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fcd391-8249-448c-b2a9-71c5f218824f None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:57.426142 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-36fcd391-8249-448c-b2a9-71c5f218824f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:57.591577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fcd391-8249-448c-b2a9-71c5f218824f None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:57.596697 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0647a4ad-3005-4c5a-868b-83cf365b36d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:57.597690 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647a4ad-3005-4c5a-868b-83cf365b36d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:57.598023 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647a4ad-3005-4c5a-868b-83cf365b36d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:57.598365 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647a4ad-3005-4c5a-868b-83cf365b36d6 admin admin] PATH_INFO: `/v3/projects/3a75f088b63d48eb94701792206e1a8d` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:57.599700 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0647a4ad-3005-4c5a-868b-83cf365b36d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a75f088b63d48eb94701792206e1a8d)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:57.602144 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0647a4ad-3005-4c5a-868b-83cf365b36d6 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:57.703163 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-0647a4ad-3005-4c5a-868b-83cf365b36d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a75f088b63d48eb94701792206e1a8d', 'request_id': 'req-0647a4ad-3005-4c5a-868b-83cf365b36d6'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:57.718399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-0647a4ad-3005-4c5a-868b-83cf365b36d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a75f088b63d48eb94701792206e1a8d', 'request_id': 'req-0647a4ad-3005-4c5a-868b-83cf365b36d6'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:57.735647 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-0647a4ad-3005-4c5a-868b-83cf365b36d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a75f088b63d48eb94701792206e1a8d', 'request_id': 'req-0647a4ad-3005-4c5a-868b-83cf365b36d6'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:57.756514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-0647a4ad-3005-4c5a-868b-83cf365b36d6 admin admin] The token cache is being invalidate because project 3a75f088b63d48eb94701792206e1a8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:15:57.756810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-0647a4ad-3005-4c5a-868b-83cf365b36d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:57.759663 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-0647a4ad-3005-4c5a-868b-83cf365b36d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:15:57.765544 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 833/1654] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:15:57 2022] DELETE /identity/v3/projects/3a75f088b63d48eb94701792206e1a8d => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:15:57.771246 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdfbf2c-8cdd-47b3-98b9-1559daf8d8e8 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:57.772163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdfbf2c-8cdd-47b3-98b9-1559daf8d8e8 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:57.772656 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdfbf2c-8cdd-47b3-98b9-1559daf8d8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:57.813556 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-abdfbf2c-8cdd-47b3-98b9-1559daf8d8e8 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:57.815356 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-abdfbf2c-8cdd-47b3-98b9-1559daf8d8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:57.986694 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 822/1655] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:15:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:15:57.986978 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27322601-25b5-49f3-bc91-ed80abe8c7c6 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:57.987274 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27322601-25b5-49f3-bc91-ed80abe8c7c6 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:57.987554 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27322601-25b5-49f3-bc91-ed80abe8c7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:58.051143 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-27322601-25b5-49f3-bc91-ed80abe8c7c6 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:58.052431 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-27322601-25b5-49f3-bc91-ed80abe8c7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:58.084464 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 834/1656] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:15:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:58.089463 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdfbf2c-8cdd-47b3-98b9-1559daf8d8e8 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:58.110905 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdfbf2c-8cdd-47b3-98b9-1559daf8d8e8 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:58.122786 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f13d61-b381-4a1f-9fcb-76ffc8d9a093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:58.123480 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f13d61-b381-4a1f-9fcb-76ffc8d9a093 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:58.123931 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f13d61-b381-4a1f-9fcb-76ffc8d9a093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:58.124345 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f13d61-b381-4a1f-9fcb-76ffc8d9a093 admin admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:58.125300 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f13d61-b381-4a1f-9fcb-76ffc8d9a093 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:15:58.125749 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f13d61-b381-4a1f-9fcb-76ffc8d9a093 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:58.127610 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f13d61-b381-4a1f-9fcb-76ffc8d9a093 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:58.169601 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27322601-25b5-49f3-bc91-ed80abe8c7c6 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:58.174035 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 823/1657] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:15:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:15:58.189904 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27322601-25b5-49f3-bc91-ed80abe8c7c6 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:58.203256 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9010c6d-3bca-4afb-b823-ed522a69c05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:58.203955 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9010c6d-3bca-4afb-b823-ed522a69c05f admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:58.204241 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9010c6d-3bca-4afb-b823-ed522a69c05f admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:58.204527 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9010c6d-3bca-4afb-b823-ed522a69c05f admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:58.206466 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9010c6d-3bca-4afb-b823-ed522a69c05f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:58.213040 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9010c6d-3bca-4afb-b823-ed522a69c05f admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:58.299163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f13d61-b381-4a1f-9fcb-76ffc8d9a093 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:58.305035 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 835/1658] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:15:58 2022] POST /identity/v3/projects => generated 377 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:58.313523 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f13d61-b381-4a1f-9fcb-76ffc8d9a093 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:58.471329 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed4e05b-7212-4016-850d-2a214ba3e9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:58.474043 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4e05b-7212-4016-850d-2a214ba3e9cf admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:58.476051 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4e05b-7212-4016-850d-2a214ba3e9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:58.476617 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4e05b-7212-4016-850d-2a214ba3e9cf admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:58.481974 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed4e05b-7212-4016-850d-2a214ba3e9cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:58.486810 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed4e05b-7212-4016-850d-2a214ba3e9cf admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:58.522030 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9010c6d-3bca-4afb-b823-ed522a69c05f admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:58.533181 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 824/1659] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:15:58 2022] POST /identity/v3/users => generated 289 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:58.535245 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9010c6d-3bca-4afb-b823-ed522a69c05f admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:58.540477 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a07905-8046-4125-94b1-9072d53b9f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:58.541666 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a07905-8046-4125-94b1-9072d53b9f77 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:58.541991 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a07905-8046-4125-94b1-9072d53b9f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:58.542426 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a07905-8046-4125-94b1-9072d53b9f77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:58.543199 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a07905-8046-4125-94b1-9072d53b9f77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:58.545039 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a07905-8046-4125-94b1-9072d53b9f77 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:58.552981 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4eb19-c175-4e3b-98d7-0dab20444737 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:58.553278 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4eb19-c175-4e3b-98d7-0dab20444737 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:58.553557 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4eb19-c175-4e3b-98d7-0dab20444737 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:58.555607 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 825/1660] 10.222.0.13 () {54 vars in 912 bytes} [Mon Jun 27 21:15:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:15:58.569207 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce4eb19-c175-4e3b-98d7-0dab20444737 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:58.581243 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 836/1661] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:58.584586 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd92d4b-0fea-40d9-8177-3529a2002be5 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:58.584819 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd92d4b-0fea-40d9-8177-3529a2002be5 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:58.586246 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd92d4b-0fea-40d9-8177-3529a2002be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:58.592739 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce4eb19-c175-4e3b-98d7-0dab20444737 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:58.608010 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35087c63-7209-4e1c-8a8c-a845511acd6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:58.610037 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35087c63-7209-4e1c-8a8c-a845511acd6b admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:58.610338 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35087c63-7209-4e1c-8a8c-a845511acd6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:58.610906 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35087c63-7209-4e1c-8a8c-a845511acd6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:58.615477 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35087c63-7209-4e1c-8a8c-a845511acd6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:58.618549 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35087c63-7209-4e1c-8a8c-a845511acd6b admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:58.659549 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 826/1662] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:15:58.660562 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35087c63-7209-4e1c-8a8c-a845511acd6b admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:58.687334 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35087c63-7209-4e1c-8a8c-a845511acd6b admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:58.699887 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-1bd92d4b-0fea-40d9-8177-3529a2002be5 None None] MFA Rules not processed for user `2a4e105858324391b5142d57e34aff9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:58.701287 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-1bd92d4b-0fea-40d9-8177-3529a2002be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:58.705855 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a65e764-1919-4ca0-a58b-d1e00b208de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:58.706937 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65e764-1919-4ca0-a58b-d1e00b208de8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:58.707223 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65e764-1919-4ca0-a58b-d1e00b208de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:58.707501 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65e764-1919-4ca0-a58b-d1e00b208de8 admin admin] PATH_INFO: `/v3/projects/facae1333bf447d792719920c88e54d0/users/eb061d0e9a1e490f93c319ec554f7599/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:58.719595 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:58.752472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a65e764-1919-4ca0-a58b-d1e00b208de8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=facae1333bf447d792719920c88e54d0, user_id=eb061d0e9a1e490f93c319ec554f7599, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:58.755317 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a65e764-1919-4ca0-a58b-d1e00b208de8 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:58.844300 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 827/1663] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:15:58 2022] PUT /identity/v3/projects/facae1333bf447d792719920c88e54d0/users/eb061d0e9a1e490f93c319ec554f7599/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:15:58.848419 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731d126-6445-49af-9eb6-becd372b3ec1 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:58.848717 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731d126-6445-49af-9eb6-becd372b3ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:58.853124 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731d126-6445-49af-9eb6-becd372b3ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:58.894102 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 837/1664] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Jun 27 21:15:58 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:58.907288 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd92d4b-0fea-40d9-8177-3529a2002be5 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:58.908738 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-1bd92d4b-0fea-40d9-8177-3529a2002be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:58.926670 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd92d4b-0fea-40d9-8177-3529a2002be5 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:58.929423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:15:58.965117 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-8731d126-6445-49af-9eb6-becd372b3ec1 None None] MFA Rules not processed for user `eb061d0e9a1e490f93c319ec554f7599`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:15:58.973202 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-8731d126-6445-49af-9eb6-becd372b3ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:15:59.154938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6100aa5c-8e56-44f4-a137-8da8d0471049 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:59.155406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100aa5c-8e56-44f4-a137-8da8d0471049 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:59.155684 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100aa5c-8e56-44f4-a137-8da8d0471049 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:59.156162 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6100aa5c-8e56-44f4-a137-8da8d0471049 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:59.178348 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6100aa5c-8e56-44f4-a137-8da8d0471049 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:59.179862 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6100aa5c-8e56-44f4-a137-8da8d0471049 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:59.220134 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 838/1665] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:15:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:15:59.242438 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6100aa5c-8e56-44f4-a137-8da8d0471049 service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:59.243750 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 828/1666] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:15:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:15:59.402499 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6100aa5c-8e56-44f4-a137-8da8d0471049 service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:15:59.417611 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01391b9-df0a-458b-893e-30e66a80b007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:15:59.427967 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01391b9-df0a-458b-893e-30e66a80b007 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:15:59.429413 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01391b9-df0a-458b-893e-30e66a80b007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:15:59.430226 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01391b9-df0a-458b-893e-30e66a80b007 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:15:59.435166 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01391b9-df0a-458b-893e-30e66a80b007 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:15:59.469465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01391b9-df0a-458b-893e-30e66a80b007 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:15:59.599181 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731d126-6445-49af-9eb6-becd372b3ec1 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:15:59.612116 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 839/1667] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:15:59 2022] POST /identity/v3/projects => generated 375 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:15:59.633598 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731d126-6445-49af-9eb6-becd372b3ec1 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:00.082597 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ea7f7-5b17-4c2f-92eb-e0e35bc95036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:00.083920 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ea7f7-5b17-4c2f-92eb-e0e35bc95036 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:00.084408 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ea7f7-5b17-4c2f-92eb-e0e35bc95036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:00.084780 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ea7f7-5b17-4c2f-92eb-e0e35bc95036 admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:00.085684 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ea7f7-5b17-4c2f-92eb-e0e35bc95036 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:00.088635 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ea7f7-5b17-4c2f-92eb-e0e35bc95036 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:00.137925 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 829/1668] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:15:59 2022] POST /identity/v3/users => generated 294 bytes in 540 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:00.144240 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01391b9-df0a-458b-893e-30e66a80b007 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:00.175865 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01391b9-df0a-458b-893e-30e66a80b007 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:00.216249 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc764f82-fcbc-40cb-8877-0a13f1f39aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:00.216993 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc764f82-fcbc-40cb-8877-0a13f1f39aa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:00.217283 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc764f82-fcbc-40cb-8877-0a13f1f39aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:00.217557 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc764f82-fcbc-40cb-8877-0a13f1f39aa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:00.218349 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc764f82-fcbc-40cb-8877-0a13f1f39aa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:00.229477 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc764f82-fcbc-40cb-8877-0a13f1f39aa1 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:00.290353 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 840/1669] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:00.300301 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ea7f7-5b17-4c2f-92eb-e0e35bc95036 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:00.359636 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ea7f7-5b17-4c2f-92eb-e0e35bc95036 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:00.381373 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163b5c19-d989-4500-8505-6d9f55780b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:00.382688 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b5c19-d989-4500-8505-6d9f55780b8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:00.388760 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b5c19-d989-4500-8505-6d9f55780b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:00.389013 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b5c19-d989-4500-8505-6d9f55780b8f admin admin] PATH_INFO: `/v3/projects/ca9ba1db2acb4da79b70382068265678/users/99741385667148aa91e657ebdcd92b66/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:00.442863 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163b5c19-d989-4500-8505-6d9f55780b8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca9ba1db2acb4da79b70382068265678, user_id=99741385667148aa91e657ebdcd92b66, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:00.446337 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163b5c19-d989-4500-8505-6d9f55780b8f admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:00.478974 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 830/1670] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:00 2022] PUT /identity/v3/projects/ca9ba1db2acb4da79b70382068265678/users/99741385667148aa91e657ebdcd92b66/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:00.484250 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc764f82-fcbc-40cb-8877-0a13f1f39aa1 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:00.503258 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc764f82-fcbc-40cb-8877-0a13f1f39aa1 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:00.608207 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:00.684614 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ff00a-5299-42d5-af92-bf2877c3116a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:00.686763 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ff00a-5299-42d5-af92-bf2877c3116a admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:00.687267 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ff00a-5299-42d5-af92-bf2877c3116a admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:00.687544 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ff00a-5299-42d5-af92-bf2877c3116a admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:00.688664 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988ff00a-5299-42d5-af92-bf2877c3116a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:00.690676 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988ff00a-5299-42d5-af92-bf2877c3116a admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:00.739807 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 841/1671] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:00.744483 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163b5c19-d989-4500-8505-6d9f55780b8f admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:00.764780 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163b5c19-d989-4500-8505-6d9f55780b8f admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:00.777805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f721a173-03be-4ad8-bb7a-0070e7d5600c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:00.782314 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721a173-03be-4ad8-bb7a-0070e7d5600c admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:00.782673 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721a173-03be-4ad8-bb7a-0070e7d5600c admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:00.782956 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721a173-03be-4ad8-bb7a-0070e7d5600c admin admin] PATH_INFO: `/v3/projects/ca9ba1db2acb4da79b70382068265678/users/99741385667148aa91e657ebdcd92b66/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:00.789440 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f721a173-03be-4ad8-bb7a-0070e7d5600c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca9ba1db2acb4da79b70382068265678, user_id=99741385667148aa91e657ebdcd92b66, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:00.799809 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f721a173-03be-4ad8-bb7a-0070e7d5600c admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:00.823289 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16afe14d-cde1-4b71-9852-dc84ca67ccf5 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:00.823687 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16afe14d-cde1-4b71-9852-dc84ca67ccf5 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:00.823977 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16afe14d-cde1-4b71-9852-dc84ca67ccf5 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:00.825748 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 842/1672] 10.222.0.13 () {54 vars in 960 bytes} [Mon Jun 27 21:16:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:16:00.834967 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491121c-8a93-4391-ae7f-7b9f782119de None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:00.835252 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491121c-8a93-4391-ae7f-7b9f782119de None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:00.835585 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491121c-8a93-4391-ae7f-7b9f782119de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:00.870200 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-9491121c-8a93-4391-ae7f-7b9f782119de None None] MFA Rules not processed for user `0437c45d72cd4a38b9d48ee6f50946fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:00.880048 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-9491121c-8a93-4391-ae7f-7b9f782119de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:00.881298 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 831/1673] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:00 2022] PUT /identity/v3/projects/ca9ba1db2acb4da79b70382068265678/users/99741385667148aa91e657ebdcd92b66/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:00.883220 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a4d13-994a-4e79-9c3d-dca389d7139e None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:00.883680 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a4d13-994a-4e79-9c3d-dca389d7139e None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:00.884093 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a4d13-994a-4e79-9c3d-dca389d7139e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:00.962507 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-991a4d13-994a-4e79-9c3d-dca389d7139e None None] MFA Rules not processed for user `99741385667148aa91e657ebdcd92b66`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:00.964316 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-991a4d13-994a-4e79-9c3d-dca389d7139e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:01.064953 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:01.160634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 832/1674] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:00 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:01.161001 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 843/1675] 10.222.0.13 () {58 vars in 1057 bytes} [Mon Jun 27 21:16:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:01.164481 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991a4d13-994a-4e79-9c3d-dca389d7139e None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:01.170381 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9491121c-8a93-4391-ae7f-7b9f782119de None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:01.172148 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-9491121c-8a93-4391-ae7f-7b9f782119de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:01.203632 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991a4d13-994a-4e79-9c3d-dca389d7139e None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:01.223371 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c19bd0-9220-4470-ab6f-ce79c23f4335 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0437c45d72cd4a38b9d48ee6f50946fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:01.224161 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c19bd0-9220-4470-ab6f-ce79c23f4335 service placement] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:01.226365 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c19bd0-9220-4470-ab6f-ce79c23f4335 service placement] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:01.226769 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c19bd0-9220-4470-ab6f-ce79c23f4335 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:01.231148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-b6c19bd0-9220-4470-ab6f-ce79c23f4335 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:01.308614 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c19bd0-9220-4470-ab6f-ce79c23f4335 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:01.308614 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c19bd0-9220-4470-ab6f-ce79c23f4335 service placement] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:01.347267 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 833/1676] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:01.362186 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bcfafa-45ce-4cd3-a051-30ed979d2091 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:01.366332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bcfafa-45ce-4cd3-a051-30ed979d2091 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:01.368248 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bcfafa-45ce-4cd3-a051-30ed979d2091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:01.392584 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9491121c-8a93-4391-ae7f-7b9f782119de None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:01.409570 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f237f45-e78c-4bca-aa90-760922241b31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:01.413385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f237f45-e78c-4bca-aa90-760922241b31 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:01.413673 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f237f45-e78c-4bca-aa90-760922241b31 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:01.413947 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f237f45-e78c-4bca-aa90-760922241b31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:01.434282 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-e4bcfafa-45ce-4cd3-a051-30ed979d2091 None None] MFA Rules not processed for user `a0408ee5961249d1bd296a6ef1a030a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:01.441000 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-e4bcfafa-45ce-4cd3-a051-30ed979d2091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:01.576753 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f237f45-e78c-4bca-aa90-760922241b31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:01.582533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f237f45-e78c-4bca-aa90-760922241b31 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:01.629216 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 844/1677] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:01 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:16:01.631469 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 834/1678] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:03.947863 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f237f45-e78c-4bca-aa90-760922241b31 service cinder] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:03.950357 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-5f237f45-e78c-4bca-aa90-760922241b31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:04.037549 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:04.161849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f237f45-e78c-4bca-aa90-760922241b31 service cinder] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:04.186431 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0114b-921d-4248-9adb-42481db0c992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:04.188313 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0114b-921d-4248-9adb-42481db0c992 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:04.188833 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0114b-921d-4248-9adb-42481db0c992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:04.191701 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a0114b-921d-4248-9adb-42481db0c992 admin admin] PATH_INFO: `/v3/users/123912b9f9a345f5bef7c88529b37838` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:04.193864 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a0114b-921d-4248-9adb-42481db0c992 admin admin] RBAC: Authorizing `identity:delete_user(user_id=123912b9f9a345f5bef7c88529b37838)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:04.199850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a0114b-921d-4248-9adb-42481db0c992 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:04.340509 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-34a0114b-921d-4248-9adb-42481db0c992 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '123912b9f9a345f5bef7c88529b37838'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:04.358454 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-34a0114b-921d-4248-9adb-42481db0c992 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '123912b9f9a345f5bef7c88529b37838'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:04.359240 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-34a0114b-921d-4248-9adb-42481db0c992 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '123912b9f9a345f5bef7c88529b37838'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:04.359851 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-34a0114b-921d-4248-9adb-42481db0c992 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '123912b9f9a345f5bef7c88529b37838'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:04.385445 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-34a0114b-921d-4248-9adb-42481db0c992 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '123912b9f9a345f5bef7c88529b37838'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:04.467731 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 845/1679] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:16:03 2022] DELETE /identity/v3/users/123912b9f9a345f5bef7c88529b37838 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:04.475054 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370da0c8-d442-4f45-90c2-f7f2b137acc0 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:04.475054 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370da0c8-d442-4f45-90c2-f7f2b137acc0 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:04.475054 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370da0c8-d442-4f45-90c2-f7f2b137acc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:04.502896 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-370da0c8-d442-4f45-90c2-f7f2b137acc0 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:04.509892 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-370da0c8-d442-4f45-90c2-f7f2b137acc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:04.712960 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 835/1680] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:04.762021 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0114b-921d-4248-9adb-42481db0c992 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:04.764699 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-34a0114b-921d-4248-9adb-42481db0c992 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:04.952389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a0114b-921d-4248-9adb-42481db0c992 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:04.958819 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bfd17e-9dd9-4261-9993-53fed0cad490 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:04.959776 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bfd17e-9dd9-4261-9993-53fed0cad490 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:04.960204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bfd17e-9dd9-4261-9993-53fed0cad490 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:04.961288 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bfd17e-9dd9-4261-9993-53fed0cad490 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:04.983949 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bfd17e-9dd9-4261-9993-53fed0cad490 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:04.986188 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bfd17e-9dd9-4261-9993-53fed0cad490 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:05.000429 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 846/1681] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:16:05.350864 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01825a-1030-4ff1-ad9d-de411e495bb1 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:05.351449 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01825a-1030-4ff1-ad9d-de411e495bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:05.351915 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01825a-1030-4ff1-ad9d-de411e495bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:05.422060 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-3e01825a-1030-4ff1-ad9d-de411e495bb1 None None] MFA Rules not processed for user `7969c6620a0d406890fb4c15b58ab349`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:05.423533 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-3e01825a-1030-4ff1-ad9d-de411e495bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:05.591633 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:05.614112 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 836/1682] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:05.628620 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bfd17e-9dd9-4261-9993-53fed0cad490 service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:05.632309 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-f0bfd17e-9dd9-4261-9993-53fed0cad490 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:05.851242 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:05.922035 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bfd17e-9dd9-4261-9993-53fed0cad490 service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:05.941072 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b074f0f-7ca4-4dd1-b6f7-0105b07aba75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:05.944251 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b074f0f-7ca4-4dd1-b6f7-0105b07aba75 service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:05.945191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b074f0f-7ca4-4dd1-b6f7-0105b07aba75 service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:05.945930 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b074f0f-7ca4-4dd1-b6f7-0105b07aba75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:05.973069 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b074f0f-7ca4-4dd1-b6f7-0105b07aba75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:05.977953 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b074f0f-7ca4-4dd1-b6f7-0105b07aba75 service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:06.000360 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 847/1683] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:07.744513 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e01825a-1030-4ff1-ad9d-de411e495bb1 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:07.746217 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-3e01825a-1030-4ff1-ad9d-de411e495bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:07.984156 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e01825a-1030-4ff1-ad9d-de411e495bb1 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:07.984663 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b074f0f-7ca4-4dd1-b6f7-0105b07aba75 service nova] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:07.987232 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-3b074f0f-7ca4-4dd1-b6f7-0105b07aba75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:07.994324 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348cdc95-b8fa-4cb0-b31c-8aee21de4a11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:07.995335 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cdc95-b8fa-4cb0-b31c-8aee21de4a11 service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:07.995691 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cdc95-b8fa-4cb0-b31c-8aee21de4a11 service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:07.996048 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cdc95-b8fa-4cb0-b31c-8aee21de4a11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:08.001787 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-348cdc95-b8fa-4cb0-b31c-8aee21de4a11 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:08.033933 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b074f0f-7ca4-4dd1-b6f7-0105b07aba75 service nova] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:08.048834 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8792419-0993-4a81-85af-f109deb90723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:08.050039 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8792419-0993-4a81-85af-f109deb90723 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:08.050531 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8792419-0993-4a81-85af-f109deb90723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:08.051168 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8792419-0993-4a81-85af-f109deb90723 admin admin] PATH_INFO: `/v3/projects/ec2655c9c3da4ee1b96b589e336fa5ea` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:08.053724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8792419-0993-4a81-85af-f109deb90723 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec2655c9c3da4ee1b96b589e336fa5ea)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:08.058082 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8792419-0993-4a81-85af-f109deb90723 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:08.211379 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c8792419-0993-4a81-85af-f109deb90723 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec2655c9c3da4ee1b96b589e336fa5ea', 'request_id': 'req-c8792419-0993-4a81-85af-f109deb90723'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:08.235222 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c8792419-0993-4a81-85af-f109deb90723 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec2655c9c3da4ee1b96b589e336fa5ea', 'request_id': 'req-c8792419-0993-4a81-85af-f109deb90723'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:08.248443 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348cdc95-b8fa-4cb0-b31c-8aee21de4a11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:08.252476 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348cdc95-b8fa-4cb0-b31c-8aee21de4a11 service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:08.259485 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c8792419-0993-4a81-85af-f109deb90723 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec2655c9c3da4ee1b96b589e336fa5ea', 'request_id': 'req-c8792419-0993-4a81-85af-f109deb90723'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:08.304886 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 837/1684] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:07 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:08.316712 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3e3af-0616-48e7-a42f-5f198247a386 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:08.317057 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3e3af-0616-48e7-a42f-5f198247a386 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:08.317345 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3e3af-0616-48e7-a42f-5f198247a386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:08.332090 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c8792419-0993-4a81-85af-f109deb90723 admin admin] The token cache is being invalidate because project ec2655c9c3da4ee1b96b589e336fa5ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:08.334684 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c8792419-0993-4a81-85af-f109deb90723 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:08.335532 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c8792419-0993-4a81-85af-f109deb90723 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:08.343949 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 848/1685] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:07 2022] DELETE /identity/v3/projects/ec2655c9c3da4ee1b96b589e336fa5ea => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:08.356426 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a53de-c41d-4e09-bced-682f695d1196 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:08.356826 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a53de-c41d-4e09-bced-682f695d1196 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:08.357239 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a53de-c41d-4e09-bced-682f695d1196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:08.410884 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-93e3e3af-0616-48e7-a42f-5f198247a386 None None] MFA Rules not processed for user `db0e1b60083640bd89b9b4287fdb9d28`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:08.414271 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-93e3e3af-0616-48e7-a42f-5f198247a386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:08.420229 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-189a53de-c41d-4e09-bced-682f695d1196 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:08.428825 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-189a53de-c41d-4e09-bced-682f695d1196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:08.682357 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 849/1686] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:16:08.686615 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec2204-5b2c-4940-bd29-88cad5d1359c None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:08.687093 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec2204-5b2c-4940-bd29-88cad5d1359c None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:08.687474 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec2204-5b2c-4940-bd29-88cad5d1359c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:08.779392 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-5dec2204-5b2c-4940-bd29-88cad5d1359c None None] MFA Rules not processed for user `f67110d591a94537a3df6a2d835370f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:08.784413 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-5dec2204-5b2c-4940-bd29-88cad5d1359c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:08.796330 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 838/1687] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 483 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:08.826003 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb7f0f-ab36-414a-bbcd-44214454873b None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:08.827008 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb7f0f-ab36-414a-bbcd-44214454873b None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:08.828792 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cb7f0f-ab36-414a-bbcd-44214454873b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:08.967913 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-58cb7f0f-ab36-414a-bbcd-44214454873b None None] MFA Rules not processed for user `fff9f1fc84a54b83a7a384965f40c9d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:08.970066 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-58cb7f0f-ab36-414a-bbcd-44214454873b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:09.002842 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:09.027979 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 850/1688] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Jun 27 21:16:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:09.029693 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da6589-9c85-4d15-87a4-fd37bf14d1fe None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:09.030243 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da6589-9c85-4d15-87a4-fd37bf14d1fe None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:09.031395 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da6589-9c85-4d15-87a4-fd37bf14d1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:09.146565 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-07da6589-9c85-4d15-87a4-fd37bf14d1fe None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:09.154612 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-07da6589-9c85-4d15-87a4-fd37bf14d1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:09.164444 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:09.206619 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 851/1689] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:09.210644 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aacc4d-cced-4d65-a9a1-d2927dd656bb None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:09.214697 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aacc4d-cced-4d65-a9a1-d2927dd656bb None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:09.215351 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aacc4d-cced-4d65-a9a1-d2927dd656bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:09.296064 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 839/1690] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 498 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:09.297945 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cb7f0f-ab36-414a-bbcd-44214454873b None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:09.299365 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-86aacc4d-cced-4d65-a9a1-d2927dd656bb None None] MFA Rules not processed for user `db0e1b60083640bd89b9b4287fdb9d28`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:09.302073 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-86aacc4d-cced-4d65-a9a1-d2927dd656bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:09.345531 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 852/1691] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:09.346768 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aacc4d-cced-4d65-a9a1-d2927dd656bb None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:09.352466 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cb7f0f-ab36-414a-bbcd-44214454873b None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:09.358945 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-86aacc4d-cced-4d65-a9a1-d2927dd656bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:09.370443 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a197ed-05c3-4bd4-afcc-f7d22a786567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:09.372368 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a197ed-05c3-4bd4-afcc-f7d22a786567 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:09.374288 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a197ed-05c3-4bd4-afcc-f7d22a786567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:09.374680 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a197ed-05c3-4bd4-afcc-f7d22a786567 admin admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:09.375545 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a197ed-05c3-4bd4-afcc-f7d22a786567 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:16:09.375866 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a197ed-05c3-4bd4-afcc-f7d22a786567 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:09.385480 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a197ed-05c3-4bd4-afcc-f7d22a786567 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:09.432028 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 840/1692] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:16:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:16:09.439038 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a197ed-05c3-4bd4-afcc-f7d22a786567 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:09.538783 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a197ed-05c3-4bd4-afcc-f7d22a786567 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:09.558186 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f283473f-e2de-4afb-94f6-b81cf5fbce2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:09.558707 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283473f-e2de-4afb-94f6-b81cf5fbce2e admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:09.566675 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283473f-e2de-4afb-94f6-b81cf5fbce2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:09.566911 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283473f-e2de-4afb-94f6-b81cf5fbce2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:09.567903 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f283473f-e2de-4afb-94f6-b81cf5fbce2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:09.574553 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f283473f-e2de-4afb-94f6-b81cf5fbce2e admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:09.622483 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aacc4d-cced-4d65-a9a1-d2927dd656bb None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:09.645139 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e461f0-24f8-448d-a8f0-56402be8209d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:09.646212 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e461f0-24f8-448d-a8f0-56402be8209d service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:09.646702 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e461f0-24f8-448d-a8f0-56402be8209d service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:09.647174 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e461f0-24f8-448d-a8f0-56402be8209d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:09.662705 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e461f0-24f8-448d-a8f0-56402be8209d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:09.664818 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e461f0-24f8-448d-a8f0-56402be8209d service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:09.682536 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 841/1693] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:09 2022] POST /identity/v3/projects => generated 379 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:09.684999 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f283473f-e2de-4afb-94f6-b81cf5fbce2e admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:09.694958 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 853/1694] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:09.697104 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e461f0-24f8-448d-a8f0-56402be8209d service nova] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:09.701764 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-f283473f-e2de-4afb-94f6-b81cf5fbce2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:09.719953 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e461f0-24f8-448d-a8f0-56402be8209d service nova] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:09.880534 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af7569-cdba-45ac-ab46-658fdab95ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:09.881661 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af7569-cdba-45ac-ab46-658fdab95ae9 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:09.882175 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af7569-cdba-45ac-ab46-658fdab95ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:09.882656 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af7569-cdba-45ac-ab46-658fdab95ae9 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:09.883584 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76af7569-cdba-45ac-ab46-658fdab95ae9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:09.885910 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76af7569-cdba-45ac-ab46-658fdab95ae9 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:09.886597 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f283473f-e2de-4afb-94f6-b81cf5fbce2e admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:09.902997 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c864f7a-06d2-4559-8c8d-6f4e559e0f5d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bed6b231ee647d8a350da055d8e59d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:09.903454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c864f7a-06d2-4559-8c8d-6f4e559e0f5d service swift] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:09.903744 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c864f7a-06d2-4559-8c8d-6f4e559e0f5d service swift] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:09.904016 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c864f7a-06d2-4559-8c8d-6f4e559e0f5d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:09.961185 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 854/1695] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:09 2022] POST /identity/v3/users => generated 290 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:09.962519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af7569-cdba-45ac-ab46-658fdab95ae9 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:09.988610 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af7569-cdba-45ac-ab46-658fdab95ae9 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:10.018341 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c10543-994a-409a-9e89-482059057ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:10.019989 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c10543-994a-409a-9e89-482059057ee1 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:10.025356 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c10543-994a-409a-9e89-482059057ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:10.027706 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c10543-994a-409a-9e89-482059057ee1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:10.028820 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c10543-994a-409a-9e89-482059057ee1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:10.032620 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c10543-994a-409a-9e89-482059057ee1 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:10.079035 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 855/1696] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:16:10.080409 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c10543-994a-409a-9e89-482059057ee1 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:10.095449 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c864f7a-06d2-4559-8c8d-6f4e559e0f5d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:10.096688 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-e5c10543-994a-409a-9e89-482059057ee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:10.132831 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c864f7a-06d2-4559-8c8d-6f4e559e0f5d service swift] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:10.199757 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 842/1697] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Jun 27 21:16:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 21:16:10.220199 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c864f7a-06d2-4559-8c8d-6f4e559e0f5d service swift] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:10.232318 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:10.282757 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c864f7a-06d2-4559-8c8d-6f4e559e0f5d service swift] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:10.293145 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5442bb8a-ca05-4040-82e3-1e55098bc3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:10.295022 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c10543-994a-409a-9e89-482059057ee1 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:10.295228 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5442bb8a-ca05-4040-82e3-1e55098bc3d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:10.295679 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5442bb8a-ca05-4040-82e3-1e55098bc3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:10.296752 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5442bb8a-ca05-4040-82e3-1e55098bc3d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:10.298276 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5442bb8a-ca05-4040-82e3-1e55098bc3d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:10.302287 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5442bb8a-ca05-4040-82e3-1e55098bc3d4 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:10.310735 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b13c7-e324-45fb-8cee-e7098a393123 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:10.311560 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b13c7-e324-45fb-8cee-e7098a393123 service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:10.311945 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b13c7-e324-45fb-8cee-e7098a393123 service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:10.312325 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b13c7-e324-45fb-8cee-e7098a393123 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:10.350152 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 843/1698] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:10 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:10.353728 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5442bb8a-ca05-4040-82e3-1e55098bc3d4 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:10.389682 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5442bb8a-ca05-4040-82e3-1e55098bc3d4 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:10.409515 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1905f21-6ef0-4a27-a87d-0ed3bec33108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:10.413185 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1905f21-6ef0-4a27-a87d-0ed3bec33108 admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:10.413829 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1905f21-6ef0-4a27-a87d-0ed3bec33108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:10.414523 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1905f21-6ef0-4a27-a87d-0ed3bec33108 admin admin] PATH_INFO: `/v3/projects/44fd55136c794f28911fb7a357a3fcbb/users/ca56650c1bba4cc2a26b3427d1ff2902/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:10.496592 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b13c7-e324-45fb-8cee-e7098a393123 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:10.498562 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b13c7-e324-45fb-8cee-e7098a393123 service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:10.520742 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 856/1699] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:10 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:10.525705 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1905f21-6ef0-4a27-a87d-0ed3bec33108 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44fd55136c794f28911fb7a357a3fcbb, user_id=ca56650c1bba4cc2a26b3427d1ff2902, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:10.547922 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1905f21-6ef0-4a27-a87d-0ed3bec33108 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:10.621257 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:10.646362 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 844/1700] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:10 2022] PUT /identity/v3/projects/44fd55136c794f28911fb7a357a3fcbb/users/ca56650c1bba4cc2a26b3427d1ff2902/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:10.672800 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d803e5b-b412-4dd5-95d9-d7e6fb82cdf1 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:10.673267 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d803e5b-b412-4dd5-95d9-d7e6fb82cdf1 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:10.673958 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d803e5b-b412-4dd5-95d9-d7e6fb82cdf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:10.714506 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-3d803e5b-b412-4dd5-95d9-d7e6fb82cdf1 None None] MFA Rules not processed for user `ca56650c1bba4cc2a26b3427d1ff2902`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:10.733304 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-3d803e5b-b412-4dd5-95d9-d7e6fb82cdf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:11.069660 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 857/1701] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:16:11.076990 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1905f21-6ef0-4a27-a87d-0ed3bec33108 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:11.274076 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1905f21-6ef0-4a27-a87d-0ed3bec33108 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:11.288084 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e881bd11-3571-48a4-9e8d-f6d9ebb0447e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:11.289439 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e881bd11-3571-48a4-9e8d-f6d9ebb0447e admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:11.292779 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e881bd11-3571-48a4-9e8d-f6d9ebb0447e admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:11.294318 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e881bd11-3571-48a4-9e8d-f6d9ebb0447e admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:11.295274 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e881bd11-3571-48a4-9e8d-f6d9ebb0447e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:11.301560 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e881bd11-3571-48a4-9e8d-f6d9ebb0447e admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:11.389703 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 845/1702] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:11 2022] POST /identity/v3/projects => generated 381 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:11.391695 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d803e5b-b412-4dd5-95d9-d7e6fb82cdf1 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:11.408959 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d803e5b-b412-4dd5-95d9-d7e6fb82cdf1 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:11.573499 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dede835f-0a0e-4189-816e-d438edf98f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:11.576299 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede835f-0a0e-4189-816e-d438edf98f5d admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:11.576628 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede835f-0a0e-4189-816e-d438edf98f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:11.576999 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede835f-0a0e-4189-816e-d438edf98f5d admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:11.583649 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dede835f-0a0e-4189-816e-d438edf98f5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:11.592795 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dede835f-0a0e-4189-816e-d438edf98f5d admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:11.661397 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 858/1703] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:11 2022] POST /identity/v3/users => generated 297 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:16:11.662868 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e881bd11-3571-48a4-9e8d-f6d9ebb0447e admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:11.676288 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e881bd11-3571-48a4-9e8d-f6d9ebb0447e admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:11.695211 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a6e53-2456-4b6f-882d-b4f5fec62800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:11.695824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a6e53-2456-4b6f-882d-b4f5fec62800 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:11.696111 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a6e53-2456-4b6f-882d-b4f5fec62800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:11.696385 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a6e53-2456-4b6f-882d-b4f5fec62800 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:11.697407 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883a6e53-2456-4b6f-882d-b4f5fec62800 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:11.704164 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883a6e53-2456-4b6f-882d-b4f5fec62800 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:11.764158 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dede835f-0a0e-4189-816e-d438edf98f5d admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:11.771482 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 846/1704] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:16:11.806258 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dede835f-0a0e-4189-816e-d438edf98f5d admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:11.832593 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef052662-fe5c-455e-b67e-3a8039b70b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:11.833575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef052662-fe5c-455e-b67e-3a8039b70b2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:11.833860 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef052662-fe5c-455e-b67e-3a8039b70b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:11.834137 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef052662-fe5c-455e-b67e-3a8039b70b2f admin admin] PATH_INFO: `/v3/projects/07ebd6f9224f49c58b3c4ec4ba0d7c44/users/5f8252b5a1c14d108ea9be7bd2f50404/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:11.849774 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:11.884539 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef052662-fe5c-455e-b67e-3a8039b70b2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=07ebd6f9224f49c58b3c4ec4ba0d7c44, user_id=5f8252b5a1c14d108ea9be7bd2f50404, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:11.887355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef052662-fe5c-455e-b67e-3a8039b70b2f admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:11.920783 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 859/1705] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:11 2022] PUT /identity/v3/projects/07ebd6f9224f49c58b3c4ec4ba0d7c44/users/5f8252b5a1c14d108ea9be7bd2f50404/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:11.926296 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a6e53-2456-4b6f-882d-b4f5fec62800 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:11.943412 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a6e53-2456-4b6f-882d-b4f5fec62800 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:12.119460 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4fb0fe-bcce-4de2-9258-dfbc7e9c8b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:12.120891 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4fb0fe-bcce-4de2-9258-dfbc7e9c8b67 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:12.121406 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4fb0fe-bcce-4de2-9258-dfbc7e9c8b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:12.121842 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4fb0fe-bcce-4de2-9258-dfbc7e9c8b67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:12.122936 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4fb0fe-bcce-4de2-9258-dfbc7e9c8b67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:12.124737 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4fb0fe-bcce-4de2-9258-dfbc7e9c8b67 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:12.137243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 847/1706] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:16:12.142802 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef052662-fe5c-455e-b67e-3a8039b70b2f admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:12.158540 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef052662-fe5c-455e-b67e-3a8039b70b2f admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:12.177973 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db68e5-2559-46a2-9cff-ba41deb726f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:12.178949 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db68e5-2559-46a2-9cff-ba41deb726f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:12.179231 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db68e5-2559-46a2-9cff-ba41deb726f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:12.179506 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db68e5-2559-46a2-9cff-ba41deb726f6 admin admin] PATH_INFO: `/v3/projects/07ebd6f9224f49c58b3c4ec4ba0d7c44/users/5f8252b5a1c14d108ea9be7bd2f50404/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:12.181656 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5db68e5-2559-46a2-9cff-ba41deb726f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07ebd6f9224f49c58b3c4ec4ba0d7c44, user_id=5f8252b5a1c14d108ea9be7bd2f50404, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:12.184385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5db68e5-2559-46a2-9cff-ba41deb726f6 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:12.208863 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 860/1707] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:12 2022] PUT /identity/v3/projects/07ebd6f9224f49c58b3c4ec4ba0d7c44/users/5f8252b5a1c14d108ea9be7bd2f50404/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:12.230544 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e353eea-35b7-4fba-abe1-cb3ac850f5c6 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:12.231163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e353eea-35b7-4fba-abe1-cb3ac850f5c6 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:12.231574 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e353eea-35b7-4fba-abe1-cb3ac850f5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:12.308052 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-2e353eea-35b7-4fba-abe1-cb3ac850f5c6 None None] MFA Rules not processed for user `5f8252b5a1c14d108ea9be7bd2f50404`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:12.315478 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-2e353eea-35b7-4fba-abe1-cb3ac850f5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:12.663275 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:12.686885 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 848/1708] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:12 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 458 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:12.699683 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db68e5-2559-46a2-9cff-ba41deb726f6 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:12.702002 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-e5db68e5-2559-46a2-9cff-ba41deb726f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:13.055621 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db68e5-2559-46a2-9cff-ba41deb726f6 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:13.067286 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d9aef-83c6-4cb2-b2ab-17d788aea8ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:13.068248 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d9aef-83c6-4cb2-b2ab-17d788aea8ec service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:13.068532 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d9aef-83c6-4cb2-b2ab-17d788aea8ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:13.068908 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d9aef-83c6-4cb2-b2ab-17d788aea8ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:13.094600 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24d9aef-83c6-4cb2-b2ab-17d788aea8ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:13.096497 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24d9aef-83c6-4cb2-b2ab-17d788aea8ec service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:13.134909 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 861/1709] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:12 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:13.298429 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e353eea-35b7-4fba-abe1-cb3ac850f5c6 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:13.313386 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e353eea-35b7-4fba-abe1-cb3ac850f5c6 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:13.319524 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b2fc60-900f-4c9d-9618-ee03c5f5b3f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:13.320788 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b2fc60-900f-4c9d-9618-ee03c5f5b3f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:13.321315 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b2fc60-900f-4c9d-9618-ee03c5f5b3f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:13.322058 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b2fc60-900f-4c9d-9618-ee03c5f5b3f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:13.523145 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b2fc60-900f-4c9d-9618-ee03c5f5b3f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:13.528670 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b2fc60-900f-4c9d-9618-ee03c5f5b3f4 service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:13.554887 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 849/1710] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:13 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:14.572255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875cfbc-e0ac-4821-9d08-69ae7cf092fa None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:14.573364 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875cfbc-e0ac-4821-9d08-69ae7cf092fa None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:14.574194 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875cfbc-e0ac-4821-9d08-69ae7cf092fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:14.639325 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-f875cfbc-e0ac-4821-9d08-69ae7cf092fa None None] MFA Rules not processed for user `7969c6620a0d406890fb4c15b58ab349`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:14.645751 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-f875cfbc-e0ac-4821-9d08-69ae7cf092fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:14.792328 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b2fc60-900f-4c9d-9618-ee03c5f5b3f4 service cinder] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:14.793830 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-58b2fc60-900f-4c9d-9618-ee03c5f5b3f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:14.830103 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 862/1711] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:15.151984 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b2fc60-900f-4c9d-9618-ee03c5f5b3f4 service cinder] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:15.205325 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5429c3-eaa9-4ea2-9a46-21a6becdd219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:15.210972 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5429c3-eaa9-4ea2-9a46-21a6becdd219 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:15.216209 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5429c3-eaa9-4ea2-9a46-21a6becdd219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:15.218014 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5429c3-eaa9-4ea2-9a46-21a6becdd219 admin admin] PATH_INFO: `/v3/users/5223bd5ca2f546d191b9b65a8c71724a` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:15.222464 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5429c3-eaa9-4ea2-9a46-21a6becdd219 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5223bd5ca2f546d191b9b65a8c71724a)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:15.227161 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5429c3-eaa9-4ea2-9a46-21a6becdd219 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:15.505762 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-ee5429c3-eaa9-4ea2-9a46-21a6becdd219 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5223bd5ca2f546d191b9b65a8c71724a'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:15.546140 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-ee5429c3-eaa9-4ea2-9a46-21a6becdd219 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5223bd5ca2f546d191b9b65a8c71724a'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:15.550439 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-ee5429c3-eaa9-4ea2-9a46-21a6becdd219 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5223bd5ca2f546d191b9b65a8c71724a'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:15.555838 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-ee5429c3-eaa9-4ea2-9a46-21a6becdd219 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5223bd5ca2f546d191b9b65a8c71724a'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:15.589538 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-ee5429c3-eaa9-4ea2-9a46-21a6becdd219 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5223bd5ca2f546d191b9b65a8c71724a'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:15.701922 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 850/1712] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:16:14 2022] DELETE /identity/v3/users/5223bd5ca2f546d191b9b65a8c71724a => generated 0 bytes in 900 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:15.702838 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6738b-0978-4e98-8389-e39afa14e7ab None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:15.703333 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6738b-0978-4e98-8389-e39afa14e7ab None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:15.703925 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6738b-0978-4e98-8389-e39afa14e7ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:15.779952 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-d1c6738b-0978-4e98-8389-e39afa14e7ab None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:15.786552 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d1c6738b-0978-4e98-8389-e39afa14e7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:16.021698 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 863/1713] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:16.168441 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5429c3-eaa9-4ea2-9a46-21a6becdd219 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:16.171088 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-ee5429c3-eaa9-4ea2-9a46-21a6becdd219 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:16.221279 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:16.454174 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338d1ce-9f11-4da2-90ea-a8685fe7f270 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:16.454462 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338d1ce-9f11-4da2-90ea-a8685fe7f270 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:16.454903 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338d1ce-9f11-4da2-90ea-a8685fe7f270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:16.479431 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5429c3-eaa9-4ea2-9a46-21a6becdd219 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:16.502028 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe4935e-9783-4f21-aa19-85164c38ee6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:16.503492 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe4935e-9783-4f21-aa19-85164c38ee6a service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:16.508701 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe4935e-9783-4f21-aa19-85164c38ee6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:16.509255 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe4935e-9783-4f21-aa19-85164c38ee6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:16.531872 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-1338d1ce-9f11-4da2-90ea-a8685fe7f270 None None] MFA Rules not processed for user `2a4e105858324391b5142d57e34aff9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:16.543621 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-1338d1ce-9f11-4da2-90ea-a8685fe7f270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:16.565565 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe4935e-9783-4f21-aa19-85164c38ee6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:16.567182 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe4935e-9783-4f21-aa19-85164c38ee6a service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:16.584889 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 864/1714] 10.222.0.33 () {58 vars in 1015 bytes} [Mon Jun 27 21:16:16 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:16.611409 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 851/1715] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:16.775596 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338d1ce-9f11-4da2-90ea-a8685fe7f270 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:16.833127 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338d1ce-9f11-4da2-90ea-a8685fe7f270 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:16.849832 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7850d781-7680-4ba5-b60e-7f8d6f82f821 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:16.851365 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7850d781-7680-4ba5-b60e-7f8d6f82f821 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:16.851650 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7850d781-7680-4ba5-b60e-7f8d6f82f821 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:16.851932 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7850d781-7680-4ba5-b60e-7f8d6f82f821 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:16.877613 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7850d781-7680-4ba5-b60e-7f8d6f82f821 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:16.886993 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7850d781-7680-4ba5-b60e-7f8d6f82f821 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:16.897103 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:16.915495 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 865/1716] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:18.263486 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a19f3-23b7-4815-a5e0-f796b0e25700 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:18.264168 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a19f3-23b7-4815-a5e0-f796b0e25700 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:18.264168 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a19f3-23b7-4815-a5e0-f796b0e25700 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:18.265791 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 852/1717] 10.222.0.13 () {54 vars in 912 bytes} [Mon Jun 27 21:16:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:16:18.272353 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb607c-5624-4cb4-b5f0-2b6f5f9c0d4c None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:18.273293 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb607c-5624-4cb4-b5f0-2b6f5f9c0d4c None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:18.273741 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb607c-5624-4cb4-b5f0-2b6f5f9c0d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:18.323352 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-39cb607c-5624-4cb4-b5f0-2b6f5f9c0d4c None None] MFA Rules not processed for user `2a4e105858324391b5142d57e34aff9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:18.324598 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-39cb607c-5624-4cb4-b5f0-2b6f5f9c0d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:18.346798 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 866/1718] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Jun 27 21:16:18 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:18.416327 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a19f3-23b7-4815-a5e0-f796b0e25700 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:18.419654 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-083a19f3-23b7-4815-a5e0-f796b0e25700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:18.458768 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a19f3-23b7-4815-a5e0-f796b0e25700 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:18.467159 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8143921-e2ff-4322-8ae6-80e004ab3535 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:18.467998 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8143921-e2ff-4322-8ae6-80e004ab3535 service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:18.471576 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8143921-e2ff-4322-8ae6-80e004ab3535 service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:18.472460 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8143921-e2ff-4322-8ae6-80e004ab3535 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:18.490759 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8143921-e2ff-4322-8ae6-80e004ab3535 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:18.496756 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8143921-e2ff-4322-8ae6-80e004ab3535 service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:18.512857 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 853/1719] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:18.793560 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c298a-f41f-4a84-9c9d-879e828d91cf None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:18.795445 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c298a-f41f-4a84-9c9d-879e828d91cf None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:18.796138 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c298a-f41f-4a84-9c9d-879e828d91cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:18.877857 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-2c9c298a-f41f-4a84-9c9d-879e828d91cf None None] MFA Rules not processed for user `ba0127dd69da407ab096773fc9e22530`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:18.881863 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-2c9c298a-f41f-4a84-9c9d-879e828d91cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:19.033492 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 867/1720] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:19.204493 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8143921-e2ff-4322-8ae6-80e004ab3535 service neutron] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:19.207499 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-d8143921-e2ff-4322-8ae6-80e004ab3535 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:19.356298 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c298a-f41f-4a84-9c9d-879e828d91cf None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:19.358745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-2c9c298a-f41f-4a84-9c9d-879e828d91cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:19.401716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c298a-f41f-4a84-9c9d-879e828d91cf None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:19.414386 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775cb89d-ad28-4d6f-a9f2-189a9d327253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:19.415313 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775cb89d-ad28-4d6f-a9f2-189a9d327253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:19.415613 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775cb89d-ad28-4d6f-a9f2-189a9d327253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:19.415921 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775cb89d-ad28-4d6f-a9f2-189a9d327253 admin admin] PATH_INFO: `/v3/projects/cd3c005f16c6425fbd61a8c40dcf89fd` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:19.420805 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775cb89d-ad28-4d6f-a9f2-189a9d327253 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd3c005f16c6425fbd61a8c40dcf89fd)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:19.425219 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8143921-e2ff-4322-8ae6-80e004ab3535 service neutron] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:19.428743 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775cb89d-ad28-4d6f-a9f2-189a9d327253 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:19.438457 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a501bb-14b4-48a3-9a39-50db019d8a83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:19.439872 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a501bb-14b4-48a3-9a39-50db019d8a83 service nova] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:19.442757 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a501bb-14b4-48a3-9a39-50db019d8a83 service nova] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:19.442757 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a501bb-14b4-48a3-9a39-50db019d8a83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:19.455265 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a501bb-14b4-48a3-9a39-50db019d8a83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:19.460016 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a501bb-14b4-48a3-9a39-50db019d8a83 service nova] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:19.492447 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 854/1721] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:19.669608 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-775cb89d-ad28-4d6f-a9f2-189a9d327253 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd3c005f16c6425fbd61a8c40dcf89fd', 'request_id': 'req-775cb89d-ad28-4d6f-a9f2-189a9d327253'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:19.705213 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-775cb89d-ad28-4d6f-a9f2-189a9d327253 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd3c005f16c6425fbd61a8c40dcf89fd', 'request_id': 'req-775cb89d-ad28-4d6f-a9f2-189a9d327253'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:19.721387 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-775cb89d-ad28-4d6f-a9f2-189a9d327253 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd3c005f16c6425fbd61a8c40dcf89fd', 'request_id': 'req-775cb89d-ad28-4d6f-a9f2-189a9d327253'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:19.758799 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-775cb89d-ad28-4d6f-a9f2-189a9d327253 admin admin] The token cache is being invalidate because project cd3c005f16c6425fbd61a8c40dcf89fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:19.759669 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-775cb89d-ad28-4d6f-a9f2-189a9d327253 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:19.763347 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-775cb89d-ad28-4d6f-a9f2-189a9d327253 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:19.769436 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 868/1722] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:19 2022] DELETE /identity/v3/projects/cd3c005f16c6425fbd61a8c40dcf89fd => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:16:19.773856 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a501bb-14b4-48a3-9a39-50db019d8a83 service nova] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:19.795859 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a501bb-14b4-48a3-9a39-50db019d8a83 service nova] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:20.125423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4640249-b8b1-44ac-b62f-d0805e86226c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:20.126437 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4640249-b8b1-44ac-b62f-d0805e86226c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:20.126912 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4640249-b8b1-44ac-b62f-d0805e86226c admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:20.127198 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4640249-b8b1-44ac-b62f-d0805e86226c admin admin] PATH_INFO: `/v3/users/b1048172079c493cabce8b27e86c5a68` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:20.129061 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4640249-b8b1-44ac-b62f-d0805e86226c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1048172079c493cabce8b27e86c5a68)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:20.131504 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4640249-b8b1-44ac-b62f-d0805e86226c admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:20.186801 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf16df-6229-4c9d-8854-244869a17893 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:20.210735 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf16df-6229-4c9d-8854-244869a17893 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:20.210735 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf16df-6229-4c9d-8854-244869a17893 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:20.210735 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 869/1723] 10.222.0.13 () {54 vars in 921 bytes} [Mon Jun 27 21:16:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:16:20.229108 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748ada4-529d-4975-866d-fc663a360ba6 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:20.229969 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748ada4-529d-4975-866d-fc663a360ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:20.230470 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748ada4-529d-4975-866d-fc663a360ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:20.330762 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-9748ada4-529d-4975-866d-fc663a360ba6 None None] MFA Rules not processed for user `1366958034d540b4ba2feb421a6afabc`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:20.336473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-9748ada4-529d-4975-866d-fc663a360ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:20.584228 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:20.619824 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 870/1724] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Jun 27 21:16:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:16:20.776428 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:20.834932 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-e4640249-b8b1-44ac-b62f-d0805e86226c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1048172079c493cabce8b27e86c5a68'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:20.860372 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-e4640249-b8b1-44ac-b62f-d0805e86226c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1048172079c493cabce8b27e86c5a68'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:20.874502 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-e4640249-b8b1-44ac-b62f-d0805e86226c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1048172079c493cabce8b27e86c5a68'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:20.880339 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-e4640249-b8b1-44ac-b62f-d0805e86226c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1048172079c493cabce8b27e86c5a68'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:20.892152 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9748ada4-529d-4975-866d-fc663a360ba6 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:20.893752 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-9748ada4-529d-4975-866d-fc663a360ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:20.918621 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-e4640249-b8b1-44ac-b62f-d0805e86226c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1048172079c493cabce8b27e86c5a68'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:20.944967 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9748ada4-529d-4975-866d-fc663a360ba6 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:20.973054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c98a89a-ee41-43e8-8f9f-fb48e3b5b267 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:20.974049 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98a89a-ee41-43e8-8f9f-fb48e3b5b267 service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:20.979017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98a89a-ee41-43e8-8f9f-fb48e3b5b267 service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:20.979854 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98a89a-ee41-43e8-8f9f-fb48e3b5b267 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:20.986965 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-0c98a89a-ee41-43e8-8f9f-fb48e3b5b267 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:21.018604 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 855/1725] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:16:19 2022] DELETE /identity/v3/users/b1048172079c493cabce8b27e86c5a68 => generated 0 bytes in 1245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:21.057218 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c98a89a-ee41-43e8-8f9f-fb48e3b5b267 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:21.058998 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c98a89a-ee41-43e8-8f9f-fb48e3b5b267 service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:21.109220 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 871/1726] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:16:21.895575 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4640249-b8b1-44ac-b62f-d0805e86226c admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:21.898429 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-e4640249-b8b1-44ac-b62f-d0805e86226c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:22.007938 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4640249-b8b1-44ac-b62f-d0805e86226c admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:22.019986 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bff40a2-7e1f-4ffb-869a-ae322c8411a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:22.020994 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff40a2-7e1f-4ffb-869a-ae322c8411a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:22.021405 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff40a2-7e1f-4ffb-869a-ae322c8411a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:22.022879 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff40a2-7e1f-4ffb-869a-ae322c8411a2 admin admin] PATH_INFO: `/v3/projects/ceed20f4961c424f8e24835a581b46ac` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:22.024320 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bff40a2-7e1f-4ffb-869a-ae322c8411a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ceed20f4961c424f8e24835a581b46ac)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:22.027672 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bff40a2-7e1f-4ffb-869a-ae322c8411a2 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:22.149887 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-5bff40a2-7e1f-4ffb-869a-ae322c8411a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ceed20f4961c424f8e24835a581b46ac', 'request_id': 'req-5bff40a2-7e1f-4ffb-869a-ae322c8411a2'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:22.178395 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-5bff40a2-7e1f-4ffb-869a-ae322c8411a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ceed20f4961c424f8e24835a581b46ac', 'request_id': 'req-5bff40a2-7e1f-4ffb-869a-ae322c8411a2'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:22.203117 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-5bff40a2-7e1f-4ffb-869a-ae322c8411a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ceed20f4961c424f8e24835a581b46ac', 'request_id': 'req-5bff40a2-7e1f-4ffb-869a-ae322c8411a2'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:22.218574 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-5bff40a2-7e1f-4ffb-869a-ae322c8411a2 admin admin] The token cache is being invalidate because project ceed20f4961c424f8e24835a581b46ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:22.219144 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-5bff40a2-7e1f-4ffb-869a-ae322c8411a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:22.219975 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-5bff40a2-7e1f-4ffb-869a-ae322c8411a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:22.227927 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 856/1727] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:21 2022] DELETE /identity/v3/projects/ceed20f4961c424f8e24835a581b46ac => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:22.229551 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49632e2-0c6d-4ddc-b83b-4e643550a446 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:22.230033 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49632e2-0c6d-4ddc-b83b-4e643550a446 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:22.230110 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49632e2-0c6d-4ddc-b83b-4e643550a446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:22.260377 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-f49632e2-0c6d-4ddc-b83b-4e643550a446 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:22.261706 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-f49632e2-0c6d-4ddc-b83b-4e643550a446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:22.405302 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 872/1728] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:16:22.408753 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e377b90e-c1d3-40fd-bf1b-2e79380d3ac6 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:22.409431 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e377b90e-c1d3-40fd-bf1b-2e79380d3ac6 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:22.409898 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e377b90e-c1d3-40fd-bf1b-2e79380d3ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:22.440140 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-e377b90e-c1d3-40fd-bf1b-2e79380d3ac6 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:22.441544 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-e377b90e-c1d3-40fd-bf1b-2e79380d3ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:22.452630 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 857/1729] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:22.456700 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49632e2-0c6d-4ddc-b83b-4e643550a446 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:22.467815 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49632e2-0c6d-4ddc-b83b-4e643550a446 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:22.472822 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5bda06-ffc2-44ed-8280-aa1b3e6fbc38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:22.473580 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5bda06-ffc2-44ed-8280-aa1b3e6fbc38 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:22.474106 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5bda06-ffc2-44ed-8280-aa1b3e6fbc38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:22.474106 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5bda06-ffc2-44ed-8280-aa1b3e6fbc38 admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:22.475093 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5bda06-ffc2-44ed-8280-aa1b3e6fbc38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:16:22.475206 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5bda06-ffc2-44ed-8280-aa1b3e6fbc38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:22.476843 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5bda06-ffc2-44ed-8280-aa1b3e6fbc38 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:22.504961 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea04b74-7835-46b2-ac05-7d5933cd93f1 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:22.505186 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea04b74-7835-46b2-ac05-7d5933cd93f1 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:22.505549 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea04b74-7835-46b2-ac05-7d5933cd93f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:22.513117 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 873/1730] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:16:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:16:22.519447 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5bda06-ffc2-44ed-8280-aa1b3e6fbc38 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:22.539261 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5bda06-ffc2-44ed-8280-aa1b3e6fbc38 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:22.548644 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5168455-f868-4d36-98fd-26b82d26b9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:22.549663 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5168455-f868-4d36-98fd-26b82d26b9c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:22.551160 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5168455-f868-4d36-98fd-26b82d26b9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:22.551657 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5168455-f868-4d36-98fd-26b82d26b9c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:22.554312 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5168455-f868-4d36-98fd-26b82d26b9c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:22.558189 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5168455-f868-4d36-98fd-26b82d26b9c9 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:22.571386 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-bea04b74-7835-46b2-ac05-7d5933cd93f1 None None] MFA Rules not processed for user `fff9f1fc84a54b83a7a384965f40c9d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:22.577835 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-bea04b74-7835-46b2-ac05-7d5933cd93f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:22.618038 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 874/1731] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:22 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:16:22.622765 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5168455-f868-4d36-98fd-26b82d26b9c9 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:22.677544 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:22.756757 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 858/1732] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:22.770599 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5168455-f868-4d36-98fd-26b82d26b9c9 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:22.780566 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6523e9bc-1fdc-4b42-91cc-727658318d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:22.781911 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523e9bc-1fdc-4b42-91cc-727658318d46 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:22.782218 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523e9bc-1fdc-4b42-91cc-727658318d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:22.782495 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523e9bc-1fdc-4b42-91cc-727658318d46 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:22.792411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6523e9bc-1fdc-4b42-91cc-727658318d46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:22.798729 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6523e9bc-1fdc-4b42-91cc-727658318d46 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:22.836662 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 875/1733] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:22 2022] POST /identity/v3/users => generated 290 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:22.837831 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea04b74-7835-46b2-ac05-7d5933cd93f1 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:22.853734 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea04b74-7835-46b2-ac05-7d5933cd93f1 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:22.866619 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78235918-13dd-4f69-b81e-dbf20dbe678f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:22.867442 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78235918-13dd-4f69-b81e-dbf20dbe678f admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:22.867718 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78235918-13dd-4f69-b81e-dbf20dbe678f admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:22.867993 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78235918-13dd-4f69-b81e-dbf20dbe678f admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:22.868944 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78235918-13dd-4f69-b81e-dbf20dbe678f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:22.872084 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78235918-13dd-4f69-b81e-dbf20dbe678f admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:22.894946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 859/1734] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:22.896111 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6523e9bc-1fdc-4b42-91cc-727658318d46 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:22.910049 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6523e9bc-1fdc-4b42-91cc-727658318d46 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:22.919781 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bc7300-2d58-48cc-b176-b06df1b478ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:22.920062 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc7300-2d58-48cc-b176-b06df1b478ef admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:22.920341 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc7300-2d58-48cc-b176-b06df1b478ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:22.920618 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc7300-2d58-48cc-b176-b06df1b478ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:22.921395 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc7300-2d58-48cc-b176-b06df1b478ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:22.923757 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bc7300-2d58-48cc-b176-b06df1b478ef admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:22.942270 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 876/1735] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:22.946377 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78235918-13dd-4f69-b81e-dbf20dbe678f admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:22.969601 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78235918-13dd-4f69-b81e-dbf20dbe678f admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:22.985719 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e19405-6dc8-4664-87d2-409d04b58d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:22.988216 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e19405-6dc8-4664-87d2-409d04b58d9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:22.988584 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e19405-6dc8-4664-87d2-409d04b58d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:22.988972 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e19405-6dc8-4664-87d2-409d04b58d9b admin admin] PATH_INFO: `/v3/projects/dd77ffe1c54848809ce19edf0c199a9d/users/e1b1d460adc645bdb59b3249a4cdc23f/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:23.011755 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e19405-6dc8-4664-87d2-409d04b58d9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd77ffe1c54848809ce19edf0c199a9d, user_id=e1b1d460adc645bdb59b3249a4cdc23f, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:23.014633 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e19405-6dc8-4664-87d2-409d04b58d9b admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:23.041209 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 860/1736] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:22 2022] PUT /identity/v3/projects/dd77ffe1c54848809ce19edf0c199a9d/users/e1b1d460adc645bdb59b3249a4cdc23f/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:23.053083 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d25f61-a5c4-446d-b45f-ab71c1adcf11 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:23.053563 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d25f61-a5c4-446d-b45f-ab71c1adcf11 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:23.054126 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d25f61-a5c4-446d-b45f-ab71c1adcf11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:23.139850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-41d25f61-a5c4-446d-b45f-ab71c1adcf11 None None] MFA Rules not processed for user `e1b1d460adc645bdb59b3249a4cdc23f`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:23.141623 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-41d25f61-a5c4-446d-b45f-ab71c1adcf11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:23.176782 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:23.369738 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 877/1737] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:23.370918 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e19405-6dc8-4664-87d2-409d04b58d9b admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:23.395129 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7806295-872a-4eb8-93cb-f42a6f8c845b None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:23.395642 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7806295-872a-4eb8-93cb-f42a6f8c845b None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:23.396108 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7806295-872a-4eb8-93cb-f42a6f8c845b None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:23.407840 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 878/1738] 10.222.0.13 () {54 vars in 921 bytes} [Mon Jun 27 21:16:23 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:16:23.504147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f358e-0962-4840-a226-3964b43a976d None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:23.504520 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f358e-0962-4840-a226-3964b43a976d None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:23.504698 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f358e-0962-4840-a226-3964b43a976d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:23.585804 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-c15f358e-0962-4840-a226-3964b43a976d None None] MFA Rules not processed for user `1366958034d540b4ba2feb421a6afabc`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:23.590109 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-c15f358e-0962-4840-a226-3964b43a976d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:23.624430 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e19405-6dc8-4664-87d2-409d04b58d9b admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:23.642569 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139531b2-887f-4e40-bfe1-2b56b2918b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:23.644222 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139531b2-887f-4e40-bfe1-2b56b2918b78 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:23.645077 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139531b2-887f-4e40-bfe1-2b56b2918b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:23.645861 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139531b2-887f-4e40-bfe1-2b56b2918b78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:23.647441 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139531b2-887f-4e40-bfe1-2b56b2918b78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:23.658862 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139531b2-887f-4e40-bfe1-2b56b2918b78 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:23.748979 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 879/1739] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Jun 27 21:16:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:23.755140 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 861/1740] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:23 2022] POST /identity/v3/projects => generated 377 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:23.762311 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15f358e-0962-4840-a226-3964b43a976d None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:23.775349 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139531b2-887f-4e40-bfe1-2b56b2918b78 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:23.777700 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-139531b2-887f-4e40-bfe1-2b56b2918b78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:23.778156 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15f358e-0962-4840-a226-3964b43a976d None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:23.886718 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149dbea-3558-4c3e-8496-02682b66ff66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:23.887840 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149dbea-3558-4c3e-8496-02682b66ff66 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:23.888330 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149dbea-3558-4c3e-8496-02682b66ff66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:23.889068 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149dbea-3558-4c3e-8496-02682b66ff66 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:23.889898 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1149dbea-3558-4c3e-8496-02682b66ff66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:23.892993 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1149dbea-3558-4c3e-8496-02682b66ff66 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:23.897708 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139531b2-887f-4e40-bfe1-2b56b2918b78 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:23.908134 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea8c38a-a476-4a02-af18-b700e96cf518 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:23.909455 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8c38a-a476-4a02-af18-b700e96cf518 service nova] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:23.909986 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8c38a-a476-4a02-af18-b700e96cf518 service nova] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:23.910571 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8c38a-a476-4a02-af18-b700e96cf518 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:23.923985 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea8c38a-a476-4a02-af18-b700e96cf518 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:23.925843 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea8c38a-a476-4a02-af18-b700e96cf518 service nova] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:23.929464 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 880/1741] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:23 2022] POST /identity/v3/users => generated 289 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:23.934994 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149dbea-3558-4c3e-8496-02682b66ff66 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:23.942036 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 862/1742] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:23.958928 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149dbea-3558-4c3e-8496-02682b66ff66 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:23.969914 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab1e679-3e7b-45b4-8e2d-fb75df6e455f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:23.970672 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1e679-3e7b-45b4-8e2d-fb75df6e455f admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:23.971210 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1e679-3e7b-45b4-8e2d-fb75df6e455f admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:23.971428 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1e679-3e7b-45b4-8e2d-fb75df6e455f admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:23.972365 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab1e679-3e7b-45b4-8e2d-fb75df6e455f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:23.974823 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab1e679-3e7b-45b4-8e2d-fb75df6e455f admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:23.998254 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea8c38a-a476-4a02-af18-b700e96cf518 service nova] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:24.009062 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 881/1743] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:24.017248 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea8c38a-a476-4a02-af18-b700e96cf518 service nova] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:24.029886 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd53af4b-d2bc-4a4c-84e3-bd579138d0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:24.030812 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd53af4b-d2bc-4a4c-84e3-bd579138d0ec admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:24.031106 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd53af4b-d2bc-4a4c-84e3-bd579138d0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:24.031381 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd53af4b-d2bc-4a4c-84e3-bd579138d0ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:24.034270 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd53af4b-d2bc-4a4c-84e3-bd579138d0ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:24.041345 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd53af4b-d2bc-4a4c-84e3-bd579138d0ec admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:24.051507 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:24.071048 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 863/1744] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:24.077987 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab1e679-3e7b-45b4-8e2d-fb75df6e455f admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:24.107583 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab1e679-3e7b-45b4-8e2d-fb75df6e455f admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:24.113331 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9069cfcf-02dd-43bc-973d-1c032014361e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:24.114249 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069cfcf-02dd-43bc-973d-1c032014361e admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:24.114535 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069cfcf-02dd-43bc-973d-1c032014361e admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:24.114823 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069cfcf-02dd-43bc-973d-1c032014361e admin admin] PATH_INFO: `/v3/projects/8ce27f4a01c7408aa8b1deb1b01f489d/users/4485f56f50394d538cd912e864442e3c/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:24.154003 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9069cfcf-02dd-43bc-973d-1c032014361e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ce27f4a01c7408aa8b1deb1b01f489d, user_id=4485f56f50394d538cd912e864442e3c, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:24.157056 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9069cfcf-02dd-43bc-973d-1c032014361e admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:24.193850 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffb34d-943e-408a-ac35-061bbe1547fe None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:24.194134 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffb34d-943e-408a-ac35-061bbe1547fe None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:24.194406 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffb34d-943e-408a-ac35-061bbe1547fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:24.196418 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 882/1745] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:24 2022] PUT /identity/v3/projects/8ce27f4a01c7408aa8b1deb1b01f489d/users/4485f56f50394d538cd912e864442e3c/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:24.254503 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-f9ffb34d-943e-408a-ac35-061bbe1547fe None None] MFA Rules not processed for user `4485f56f50394d538cd912e864442e3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:24.257610 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-f9ffb34d-943e-408a-ac35-061bbe1547fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:24.501903 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 864/1746] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:24.507059 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9069cfcf-02dd-43bc-973d-1c032014361e admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:24.518131 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:24.612310 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9069cfcf-02dd-43bc-973d-1c032014361e admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:24.626871 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bcc58d-d8db-4bd1-9e23-f0cd9d79b2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:24.629169 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bcc58d-d8db-4bd1-9e23-f0cd9d79b2e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:24.629577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bcc58d-d8db-4bd1-9e23-f0cd9d79b2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:24.630815 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bcc58d-d8db-4bd1-9e23-f0cd9d79b2e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:24.631644 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bcc58d-d8db-4bd1-9e23-f0cd9d79b2e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:24.637236 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bcc58d-d8db-4bd1-9e23-f0cd9d79b2e0 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:24.745249 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 883/1747] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:24 2022] POST /identity/v3/projects => generated 377 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:24.747646 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ffb34d-943e-408a-ac35-061bbe1547fe None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:24.844744 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ffb34d-943e-408a-ac35-061bbe1547fe None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:25.131345 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2727c22d-df48-4dbf-96fc-620a958c5de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:25.132112 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2727c22d-df48-4dbf-96fc-620a958c5de2 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:25.132344 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2727c22d-df48-4dbf-96fc-620a958c5de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:25.132593 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2727c22d-df48-4dbf-96fc-620a958c5de2 admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:25.137833 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2727c22d-df48-4dbf-96fc-620a958c5de2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:25.142464 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2727c22d-df48-4dbf-96fc-620a958c5de2 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:25.239924 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bcc58d-d8db-4bd1-9e23-f0cd9d79b2e0 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:25.240918 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 865/1748] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:24 2022] POST /identity/v3/users => generated 295 bytes in 493 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:25.264879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bcc58d-d8db-4bd1-9e23-f0cd9d79b2e0 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:25.286619 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04781ac-61e7-4ee9-819f-116f4b6ae3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:25.290971 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04781ac-61e7-4ee9-819f-116f4b6ae3ab admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:25.291520 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04781ac-61e7-4ee9-819f-116f4b6ae3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:25.291897 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04781ac-61e7-4ee9-819f-116f4b6ae3ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:25.292789 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04781ac-61e7-4ee9-819f-116f4b6ae3ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:25.299041 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04781ac-61e7-4ee9-819f-116f4b6ae3ab admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:25.337457 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 884/1749] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:25.337722 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2727c22d-df48-4dbf-96fc-620a958c5de2 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:25.359544 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2727c22d-df48-4dbf-96fc-620a958c5de2 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:25.370129 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa4099-1832-4e92-afe3-cb397d348164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:25.371752 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa4099-1832-4e92-afe3-cb397d348164 admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:25.372298 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa4099-1832-4e92-afe3-cb397d348164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:25.372854 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa4099-1832-4e92-afe3-cb397d348164 admin admin] PATH_INFO: `/v3/projects/004b92123a9c4dd5af45d06b16bd03b9/users/30efb14fb6a64f8495fe3c2999a321c9/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:25.431716 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fa4099-1832-4e92-afe3-cb397d348164 admin admin] RBAC: Authorizing `identity:create_grant(project_id=004b92123a9c4dd5af45d06b16bd03b9, user_id=30efb14fb6a64f8495fe3c2999a321c9, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:25.436809 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fa4099-1832-4e92-afe3-cb397d348164 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:25.471163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 866/1750] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:25 2022] PUT /identity/v3/projects/004b92123a9c4dd5af45d06b16bd03b9/users/30efb14fb6a64f8495fe3c2999a321c9/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:25.474736 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04781ac-61e7-4ee9-819f-116f4b6ae3ab admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:25.491209 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04781ac-61e7-4ee9-819f-116f4b6ae3ab admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:25.760808 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb19fa5-461b-4486-90ea-f7f005910dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:25.762520 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb19fa5-461b-4486-90ea-f7f005910dc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:25.764693 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb19fa5-461b-4486-90ea-f7f005910dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:25.765228 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb19fa5-461b-4486-90ea-f7f005910dc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:25.767049 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb19fa5-461b-4486-90ea-f7f005910dc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:25.769152 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb19fa5-461b-4486-90ea-f7f005910dc5 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:25.795309 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 885/1751] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:25.799822 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa4099-1832-4e92-afe3-cb397d348164 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:25.825312 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa4099-1832-4e92-afe3-cb397d348164 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:25.852622 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e307fe3b-3091-471c-84d9-991cf901731b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:25.853864 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e307fe3b-3091-471c-84d9-991cf901731b admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:25.854195 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e307fe3b-3091-471c-84d9-991cf901731b admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:25.854590 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e307fe3b-3091-471c-84d9-991cf901731b admin admin] PATH_INFO: `/v3/projects/004b92123a9c4dd5af45d06b16bd03b9/users/30efb14fb6a64f8495fe3c2999a321c9/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:25.856958 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e307fe3b-3091-471c-84d9-991cf901731b admin admin] RBAC: Authorizing `identity:create_grant(project_id=004b92123a9c4dd5af45d06b16bd03b9, user_id=30efb14fb6a64f8495fe3c2999a321c9, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:25.860044 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e307fe3b-3091-471c-84d9-991cf901731b admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:25.903736 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 867/1752] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:25 2022] PUT /identity/v3/projects/004b92123a9c4dd5af45d06b16bd03b9/users/30efb14fb6a64f8495fe3c2999a321c9/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:25.906059 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5fd621-2648-4c19-b3e0-bff7bb110466 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:25.906421 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5fd621-2648-4c19-b3e0-bff7bb110466 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:25.906710 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5fd621-2648-4c19-b3e0-bff7bb110466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:26.011605 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-ff5fd621-2648-4c19-b3e0-bff7bb110466 None None] MFA Rules not processed for user `30efb14fb6a64f8495fe3c2999a321c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:26.017219 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ff5fd621-2648-4c19-b3e0-bff7bb110466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:26.188821 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:26.204205 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 886/1753] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:25 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:26.218124 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e307fe3b-3091-471c-84d9-991cf901731b admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:26.221213 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-e307fe3b-3091-471c-84d9-991cf901731b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:26.345573 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:26.390812 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e307fe3b-3091-471c-84d9-991cf901731b admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:26.400148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bda1e7-42d9-4241-bbc1-92cd39685bf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:26.403512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bda1e7-42d9-4241-bbc1-92cd39685bf2 service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:26.404027 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bda1e7-42d9-4241-bbc1-92cd39685bf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:26.404498 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bda1e7-42d9-4241-bbc1-92cd39685bf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:26.420423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bda1e7-42d9-4241-bbc1-92cd39685bf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:26.433691 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bda1e7-42d9-4241-bbc1-92cd39685bf2 service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:26.445739 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 868/1754] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:26 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:26.502647 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5fd621-2648-4c19-b3e0-bff7bb110466 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:26.506140 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ff5fd621-2648-4c19-b3e0-bff7bb110466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:26.520030 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5fd621-2648-4c19-b3e0-bff7bb110466 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:26.524201 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f8bf3-62cb-4c0a-b3f9-d992686c4fe5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:26.525887 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f8bf3-62cb-4c0a-b3f9-d992686c4fe5 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:26.526460 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f8bf3-62cb-4c0a-b3f9-d992686c4fe5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:26.526937 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f8bf3-62cb-4c0a-b3f9-d992686c4fe5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:26.646884 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad371a8-0db1-4775-9cc8-2081b81a3002 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:26.647175 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad371a8-0db1-4775-9cc8-2081b81a3002 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:26.647456 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad371a8-0db1-4775-9cc8-2081b81a3002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:26.651692 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7f8bf3-62cb-4c0a-b3f9-d992686c4fe5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:26.653636 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7f8bf3-62cb-4c0a-b3f9-d992686c4fe5 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:26.668097 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 887/1755] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:26 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:26.722050 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-aad371a8-0db1-4775-9cc8-2081b81a3002 None None] MFA Rules not processed for user `ba0127dd69da407ab096773fc9e22530`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:26.723424 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-aad371a8-0db1-4775-9cc8-2081b81a3002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:26.914442 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 869/1756] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:16:29.954691 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f8bf3-62cb-4c0a-b3f9-d992686c4fe5 service cinder] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:29.957683 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-fc7f8bf3-62cb-4c0a-b3f9-d992686c4fe5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:30.068622 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f8bf3-62cb-4c0a-b3f9-d992686c4fe5 service cinder] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:30.083137 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dbd325-a6ff-4cb0-85ac-92c99a855087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:30.087651 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbd325-a6ff-4cb0-85ac-92c99a855087 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:30.088800 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbd325-a6ff-4cb0-85ac-92c99a855087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:30.089323 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbd325-a6ff-4cb0-85ac-92c99a855087 admin admin] PATH_INFO: `/v3/users/ca56650c1bba4cc2a26b3427d1ff2902` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:30.091237 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dbd325-a6ff-4cb0-85ac-92c99a855087 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca56650c1bba4cc2a26b3427d1ff2902)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:30.094790 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dbd325-a6ff-4cb0-85ac-92c99a855087 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:30.212681 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-43dbd325-a6ff-4cb0-85ac-92c99a855087 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca56650c1bba4cc2a26b3427d1ff2902'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:30.237341 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-43dbd325-a6ff-4cb0-85ac-92c99a855087 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca56650c1bba4cc2a26b3427d1ff2902'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:30.238691 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-43dbd325-a6ff-4cb0-85ac-92c99a855087 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca56650c1bba4cc2a26b3427d1ff2902'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:30.239671 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-43dbd325-a6ff-4cb0-85ac-92c99a855087 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca56650c1bba4cc2a26b3427d1ff2902'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:30.256242 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-43dbd325-a6ff-4cb0-85ac-92c99a855087 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca56650c1bba4cc2a26b3427d1ff2902'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:30.355982 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 888/1757] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:16:29 2022] DELETE /identity/v3/users/ca56650c1bba4cc2a26b3427d1ff2902 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:30.360393 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb655e5a-4f0c-4859-8765-1b1d61a621d7 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:30.361642 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb655e5a-4f0c-4859-8765-1b1d61a621d7 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:30.362163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb655e5a-4f0c-4859-8765-1b1d61a621d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:30.448555 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-cb655e5a-4f0c-4859-8765-1b1d61a621d7 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:30.460560 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-cb655e5a-4f0c-4859-8765-1b1d61a621d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:30.579224 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 870/1758] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:30.769481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dbd325-a6ff-4cb0-85ac-92c99a855087 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:30.771597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-43dbd325-a6ff-4cb0-85ac-92c99a855087 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:30.825495 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:31.012992 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dbd325-a6ff-4cb0-85ac-92c99a855087 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:31.028548 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d8e06-46c7-4c7c-abd9-29902c50e2de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:31.032488 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d8e06-46c7-4c7c-abd9-29902c50e2de service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:31.038464 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d8e06-46c7-4c7c-abd9-29902c50e2de service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:31.043382 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d8e06-46c7-4c7c-abd9-29902c50e2de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:31.086023 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773d8e06-46c7-4c7c-abd9-29902c50e2de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:31.088668 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773d8e06-46c7-4c7c-abd9-29902c50e2de service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:31.111084 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 889/1759] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:31.731745 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb655e5a-4f0c-4859-8765-1b1d61a621d7 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:31.734801 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-cb655e5a-4f0c-4859-8765-1b1d61a621d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:31.751933 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb655e5a-4f0c-4859-8765-1b1d61a621d7 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:31.758143 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3646494-5bc7-463b-ad82-e6faf795f828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:31.759464 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3646494-5bc7-463b-ad82-e6faf795f828 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:31.760011 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3646494-5bc7-463b-ad82-e6faf795f828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:31.760529 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3646494-5bc7-463b-ad82-e6faf795f828 admin admin] PATH_INFO: `/v3/users/e1b1d460adc645bdb59b3249a4cdc23f` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:31.761891 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3646494-5bc7-463b-ad82-e6faf795f828 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1b1d460adc645bdb59b3249a4cdc23f)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:31.764385 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3646494-5bc7-463b-ad82-e6faf795f828 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:31.841878 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-e3646494-5bc7-463b-ad82-e6faf795f828 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1b1d460adc645bdb59b3249a4cdc23f'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:31.864028 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-e3646494-5bc7-463b-ad82-e6faf795f828 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1b1d460adc645bdb59b3249a4cdc23f'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:31.886840 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-e3646494-5bc7-463b-ad82-e6faf795f828 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1b1d460adc645bdb59b3249a4cdc23f'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:31.890448 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-e3646494-5bc7-463b-ad82-e6faf795f828 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1b1d460adc645bdb59b3249a4cdc23f'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:31.909257 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-e3646494-5bc7-463b-ad82-e6faf795f828 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1b1d460adc645bdb59b3249a4cdc23f'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:31.960315 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 871/1760] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:16:31 2022] DELETE /identity/v3/users/e1b1d460adc645bdb59b3249a4cdc23f => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:31.961316 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b6a17-10d0-40e2-9909-0b065c38ff2f None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:31.961614 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b6a17-10d0-40e2-9909-0b065c38ff2f None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:31.961905 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b6a17-10d0-40e2-9909-0b065c38ff2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:32.006974 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-c78b6a17-10d0-40e2-9909-0b065c38ff2f None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:32.023471 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-c78b6a17-10d0-40e2-9909-0b065c38ff2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:32.258122 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3646494-5bc7-463b-ad82-e6faf795f828 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:32.261290 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 890/1761] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:32.269547 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-e3646494-5bc7-463b-ad82-e6faf795f828 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:32.287308 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b6a17-10d0-40e2-9909-0b065c38ff2f None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:32.289807 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-c78b6a17-10d0-40e2-9909-0b065c38ff2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:32.291409 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:32.300693 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3646494-5bc7-463b-ad82-e6faf795f828 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:32.316261 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b25fc4-823f-4eeb-9334-9b0cb833a0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:32.317766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b25fc4-823f-4eeb-9334-9b0cb833a0b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:32.317766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b25fc4-823f-4eeb-9334-9b0cb833a0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:32.318165 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b25fc4-823f-4eeb-9334-9b0cb833a0b3 admin admin] PATH_INFO: `/v3/projects/44fd55136c794f28911fb7a357a3fcbb` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:32.320441 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b25fc4-823f-4eeb-9334-9b0cb833a0b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44fd55136c794f28911fb7a357a3fcbb)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:32.323412 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b25fc4-823f-4eeb-9334-9b0cb833a0b3 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:32.424828 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b6a17-10d0-40e2-9909-0b065c38ff2f None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:32.454621 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-89b25fc4-823f-4eeb-9334-9b0cb833a0b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44fd55136c794f28911fb7a357a3fcbb', 'request_id': 'req-89b25fc4-823f-4eeb-9334-9b0cb833a0b3'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:32.473365 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-89b25fc4-823f-4eeb-9334-9b0cb833a0b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44fd55136c794f28911fb7a357a3fcbb', 'request_id': 'req-89b25fc4-823f-4eeb-9334-9b0cb833a0b3'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:32.500896 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-89b25fc4-823f-4eeb-9334-9b0cb833a0b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44fd55136c794f28911fb7a357a3fcbb', 'request_id': 'req-89b25fc4-823f-4eeb-9334-9b0cb833a0b3'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:32.543836 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-89b25fc4-823f-4eeb-9334-9b0cb833a0b3 admin admin] The token cache is being invalidate because project 44fd55136c794f28911fb7a357a3fcbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:32.544458 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-89b25fc4-823f-4eeb-9334-9b0cb833a0b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:32.545568 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-89b25fc4-823f-4eeb-9334-9b0cb833a0b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:32.546561 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a85b4-3b9d-43aa-b470-ecfa5a87d758 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:32.547392 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a85b4-3b9d-43aa-b470-ecfa5a87d758 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:32.547809 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a85b4-3b9d-43aa-b470-ecfa5a87d758 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:32.547934 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a85b4-3b9d-43aa-b470-ecfa5a87d758 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:32.548748 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 872/1762] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:32 2022] DELETE /identity/v3/projects/44fd55136c794f28911fb7a357a3fcbb => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:16:32.550139 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-487a85b4-3b9d-43aa-b470-ecfa5a87d758 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:32.554038 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b25fc4-823f-4eeb-9334-9b0cb833a0b3 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:32.555587 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-89b25fc4-823f-4eeb-9334-9b0cb833a0b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:32.632884 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:32.720765 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487a85b4-3b9d-43aa-b470-ecfa5a87d758 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:32.722994 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487a85b4-3b9d-43aa-b470-ecfa5a87d758 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:32.729652 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b25fc4-823f-4eeb-9334-9b0cb833a0b3 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:32.735074 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb253e1-917b-4051-baf0-5ec0fc2e2574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:32.735784 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb253e1-917b-4051-baf0-5ec0fc2e2574 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:32.735784 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb253e1-917b-4051-baf0-5ec0fc2e2574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:32.735966 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb253e1-917b-4051-baf0-5ec0fc2e2574 admin admin] PATH_INFO: `/v3/users/5f8252b5a1c14d108ea9be7bd2f50404` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:32.737615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb253e1-917b-4051-baf0-5ec0fc2e2574 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f8252b5a1c14d108ea9be7bd2f50404)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:32.740244 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb253e1-917b-4051-baf0-5ec0fc2e2574 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:32.748167 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 891/1763] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:16:32.832293 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-adb253e1-917b-4051-baf0-5ec0fc2e2574 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f8252b5a1c14d108ea9be7bd2f50404'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:32.850246 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-adb253e1-917b-4051-baf0-5ec0fc2e2574 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f8252b5a1c14d108ea9be7bd2f50404'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:32.853876 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-adb253e1-917b-4051-baf0-5ec0fc2e2574 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f8252b5a1c14d108ea9be7bd2f50404'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:32.857037 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-adb253e1-917b-4051-baf0-5ec0fc2e2574 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f8252b5a1c14d108ea9be7bd2f50404'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:32.874016 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-adb253e1-917b-4051-baf0-5ec0fc2e2574 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f8252b5a1c14d108ea9be7bd2f50404'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:32.906700 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 873/1764] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:16:32 2022] DELETE /identity/v3/users/5f8252b5a1c14d108ea9be7bd2f50404 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:33.134943 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a85b4-3b9d-43aa-b470-ecfa5a87d758 service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:33.137427 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-487a85b4-3b9d-43aa-b470-ecfa5a87d758 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:33.155435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a85b4-3b9d-43aa-b470-ecfa5a87d758 service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:33.268165 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bafec9a-7275-4df6-896f-f6bbbabb9a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:33.269250 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bafec9a-7275-4df6-896f-f6bbbabb9a52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:33.269604 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bafec9a-7275-4df6-896f-f6bbbabb9a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:33.270716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bafec9a-7275-4df6-896f-f6bbbabb9a52 admin admin] PATH_INFO: `/v3/projects/dd77ffe1c54848809ce19edf0c199a9d` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:33.272043 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bafec9a-7275-4df6-896f-f6bbbabb9a52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd77ffe1c54848809ce19edf0c199a9d)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:33.274672 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bafec9a-7275-4df6-896f-f6bbbabb9a52 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:33.331618 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6bafec9a-7275-4df6-896f-f6bbbabb9a52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd77ffe1c54848809ce19edf0c199a9d', 'request_id': 'req-6bafec9a-7275-4df6-896f-f6bbbabb9a52'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:33.339004 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6bafec9a-7275-4df6-896f-f6bbbabb9a52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd77ffe1c54848809ce19edf0c199a9d', 'request_id': 'req-6bafec9a-7275-4df6-896f-f6bbbabb9a52'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:33.347080 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6bafec9a-7275-4df6-896f-f6bbbabb9a52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd77ffe1c54848809ce19edf0c199a9d', 'request_id': 'req-6bafec9a-7275-4df6-896f-f6bbbabb9a52'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:33.360145 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6bafec9a-7275-4df6-896f-f6bbbabb9a52 admin admin] The token cache is being invalidate because project dd77ffe1c54848809ce19edf0c199a9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:33.360512 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6bafec9a-7275-4df6-896f-f6bbbabb9a52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:33.361320 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6bafec9a-7275-4df6-896f-f6bbbabb9a52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:33.363045 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 892/1765] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:33 2022] DELETE /identity/v3/projects/dd77ffe1c54848809ce19edf0c199a9d => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:33.367453 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb253e1-917b-4051-baf0-5ec0fc2e2574 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:33.369113 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-adb253e1-917b-4051-baf0-5ec0fc2e2574 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:33.379989 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:33.386672 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bafec9a-7275-4df6-896f-f6bbbabb9a52 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:33.388328 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-6bafec9a-7275-4df6-896f-f6bbbabb9a52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:33.389732 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb253e1-917b-4051-baf0-5ec0fc2e2574 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:33.455070 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d658b9-61b8-4921-a2de-81bbe7d0ea4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:33.455992 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d658b9-61b8-4921-a2de-81bbe7d0ea4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:33.456273 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d658b9-61b8-4921-a2de-81bbe7d0ea4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:33.456547 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d658b9-61b8-4921-a2de-81bbe7d0ea4d admin admin] PATH_INFO: `/v3/users/4485f56f50394d538cd912e864442e3c` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:33.457204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:33.457815 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d658b9-61b8-4921-a2de-81bbe7d0ea4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4485f56f50394d538cd912e864442e3c)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:33.459812 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d658b9-61b8-4921-a2de-81bbe7d0ea4d admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:33.464173 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bafec9a-7275-4df6-896f-f6bbbabb9a52 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:33.470530 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0095a3-a316-4eb6-b9f5-33d318d35e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:33.471441 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0095a3-a316-4eb6-b9f5-33d318d35e96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:33.471939 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0095a3-a316-4eb6-b9f5-33d318d35e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:33.472285 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0095a3-a316-4eb6-b9f5-33d318d35e96 admin admin] PATH_INFO: `/v3/projects/07ebd6f9224f49c58b3c4ec4ba0d7c44` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:33.473766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0095a3-a316-4eb6-b9f5-33d318d35e96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07ebd6f9224f49c58b3c4ec4ba0d7c44)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:33.477160 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0095a3-a316-4eb6-b9f5-33d318d35e96 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:33.513036 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c1d658b9-61b8-4921-a2de-81bbe7d0ea4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4485f56f50394d538cd912e864442e3c'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:33.521309 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c1d658b9-61b8-4921-a2de-81bbe7d0ea4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4485f56f50394d538cd912e864442e3c'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:33.521854 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c1d658b9-61b8-4921-a2de-81bbe7d0ea4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4485f56f50394d538cd912e864442e3c'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:33.522671 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c1d658b9-61b8-4921-a2de-81bbe7d0ea4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4485f56f50394d538cd912e864442e3c'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:33.531871 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-5c0095a3-a316-4eb6-b9f5-33d318d35e96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07ebd6f9224f49c58b3c4ec4ba0d7c44', 'request_id': 'req-5c0095a3-a316-4eb6-b9f5-33d318d35e96'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:33.533114 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c1d658b9-61b8-4921-a2de-81bbe7d0ea4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4485f56f50394d538cd912e864442e3c'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:33.538286 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-5c0095a3-a316-4eb6-b9f5-33d318d35e96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07ebd6f9224f49c58b3c4ec4ba0d7c44', 'request_id': 'req-5c0095a3-a316-4eb6-b9f5-33d318d35e96'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:33.547995 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-5c0095a3-a316-4eb6-b9f5-33d318d35e96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07ebd6f9224f49c58b3c4ec4ba0d7c44', 'request_id': 'req-5c0095a3-a316-4eb6-b9f5-33d318d35e96'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:33.556405 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 874/1766] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:16:33 2022] DELETE /identity/v3/users/4485f56f50394d538cd912e864442e3c => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:33.563776 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-5c0095a3-a316-4eb6-b9f5-33d318d35e96 admin admin] The token cache is being invalidate because project 07ebd6f9224f49c58b3c4ec4ba0d7c44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:33.564074 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-5c0095a3-a316-4eb6-b9f5-33d318d35e96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:33.564761 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-5c0095a3-a316-4eb6-b9f5-33d318d35e96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:33.566433 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 893/1767] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:33 2022] DELETE /identity/v3/projects/07ebd6f9224f49c58b3c4ec4ba0d7c44 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:33.571351 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060dc267-53f0-436f-8c08-10430b160149 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:33.571755 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060dc267-53f0-436f-8c08-10430b160149 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:33.572216 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060dc267-53f0-436f-8c08-10430b160149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:33.592212 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-060dc267-53f0-436f-8c08-10430b160149 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:33.593697 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-060dc267-53f0-436f-8c08-10430b160149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:33.658109 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 875/1768] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:33.666777 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91491f3f-9a82-4901-9a34-28fff59e118e None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:33.667137 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91491f3f-9a82-4901-9a34-28fff59e118e None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:33.667419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91491f3f-9a82-4901-9a34-28fff59e118e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:33.687253 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-91491f3f-9a82-4901-9a34-28fff59e118e None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:33.688476 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-91491f3f-9a82-4901-9a34-28fff59e118e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:33.697838 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 894/1769] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:33.702164 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060dc267-53f0-436f-8c08-10430b160149 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:33.711857 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060dc267-53f0-436f-8c08-10430b160149 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:33.718304 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf632ad9-ad4f-454f-a045-e2d423a46d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:33.719187 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf632ad9-ad4f-454f-a045-e2d423a46d85 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:33.719494 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf632ad9-ad4f-454f-a045-e2d423a46d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:33.719676 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf632ad9-ad4f-454f-a045-e2d423a46d85 admin admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:33.720586 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf632ad9-ad4f-454f-a045-e2d423a46d85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:16:33.720985 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf632ad9-ad4f-454f-a045-e2d423a46d85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:33.723171 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf632ad9-ad4f-454f-a045-e2d423a46d85 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:33.737186 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 876/1770] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:16:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:16:33.741533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91491f3f-9a82-4901-9a34-28fff59e118e None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:33.752603 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91491f3f-9a82-4901-9a34-28fff59e118e None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:33.759865 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b32fc-37f9-416e-a59f-96a86b3abea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:33.760858 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b32fc-37f9-416e-a59f-96a86b3abea7 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:33.761065 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b32fc-37f9-416e-a59f-96a86b3abea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:33.761451 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b32fc-37f9-416e-a59f-96a86b3abea7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:33.762256 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172b32fc-37f9-416e-a59f-96a86b3abea7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:33.764296 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172b32fc-37f9-416e-a59f-96a86b3abea7 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:33.809768 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 895/1771] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:33 2022] POST /identity/v3/projects => generated 387 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:33.811149 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf632ad9-ad4f-454f-a045-e2d423a46d85 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:33.824809 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf632ad9-ad4f-454f-a045-e2d423a46d85 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:33.847873 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:33.902879 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc9c6a-59d6-4e3e-93eb-de3bad774155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:33.903558 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc9c6a-59d6-4e3e-93eb-de3bad774155 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:33.903838 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc9c6a-59d6-4e3e-93eb-de3bad774155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:33.904099 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc9c6a-59d6-4e3e-93eb-de3bad774155 admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:33.904818 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcc9c6a-59d6-4e3e-93eb-de3bad774155 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:33.906828 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcc9c6a-59d6-4e3e-93eb-de3bad774155 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:33.929362 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 877/1772] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:33 2022] POST /identity/v3/users => generated 294 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:33.933811 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b32fc-37f9-416e-a59f-96a86b3abea7 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:33.945044 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b32fc-37f9-416e-a59f-96a86b3abea7 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:33.950397 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59113b47-90ce-4b6b-8b10-7769c7054d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:33.951092 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59113b47-90ce-4b6b-8b10-7769c7054d56 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:33.951345 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59113b47-90ce-4b6b-8b10-7769c7054d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:33.951622 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59113b47-90ce-4b6b-8b10-7769c7054d56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:33.952364 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59113b47-90ce-4b6b-8b10-7769c7054d56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:33.954039 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59113b47-90ce-4b6b-8b10-7769c7054d56 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:33.966331 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 896/1773] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:33.970743 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc9c6a-59d6-4e3e-93eb-de3bad774155 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:33.981999 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc9c6a-59d6-4e3e-93eb-de3bad774155 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:33.987547 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee17728f-d691-4faa-9e97-3258e344016a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:33.988349 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee17728f-d691-4faa-9e97-3258e344016a admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:33.988701 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee17728f-d691-4faa-9e97-3258e344016a admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:33.991461 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee17728f-d691-4faa-9e97-3258e344016a admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:33.991461 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee17728f-d691-4faa-9e97-3258e344016a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:33.992099 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee17728f-d691-4faa-9e97-3258e344016a admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:34.006777 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 878/1774] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:34.008327 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59113b47-90ce-4b6b-8b10-7769c7054d56 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:34.020774 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59113b47-90ce-4b6b-8b10-7769c7054d56 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:34.027479 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aa1730-2271-4cdc-abe2-299d5ec1090b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:34.027479 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa1730-2271-4cdc-abe2-299d5ec1090b admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:34.027479 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa1730-2271-4cdc-abe2-299d5ec1090b admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:34.027479 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aa1730-2271-4cdc-abe2-299d5ec1090b admin admin] PATH_INFO: `/v3/projects/a5f6d7a881ba4106b694e00702c8a502/users/d6091238fa5b471481c5115703e33122/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:34.045272 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aa1730-2271-4cdc-abe2-299d5ec1090b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5f6d7a881ba4106b694e00702c8a502, user_id=d6091238fa5b471481c5115703e33122, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:34.048316 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aa1730-2271-4cdc-abe2-299d5ec1090b admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:34.064147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 897/1775] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:34 2022] PUT /identity/v3/projects/a5f6d7a881ba4106b694e00702c8a502/users/d6091238fa5b471481c5115703e33122/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:34.075297 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0124276-b7d3-4931-b429-a9bae953fbb6 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:34.075562 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0124276-b7d3-4931-b429-a9bae953fbb6 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:34.075875 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0124276-b7d3-4931-b429-a9bae953fbb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:34.094170 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aa1730-2271-4cdc-abe2-299d5ec1090b admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:34.096542 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-e7aa1730-2271-4cdc-abe2-299d5ec1090b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:34.108493 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-e0124276-b7d3-4931-b429-a9bae953fbb6 None None] MFA Rules not processed for user `d6091238fa5b471481c5115703e33122`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:34.109903 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-e0124276-b7d3-4931-b429-a9bae953fbb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:34.194386 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aa1730-2271-4cdc-abe2-299d5ec1090b admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:34.200962 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 879/1776] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:34.202153 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0124276-b7d3-4931-b429-a9bae953fbb6 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:34.206139 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54326fa-7861-46cd-87f4-42a9ecdfe3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:34.207060 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54326fa-7861-46cd-87f4-42a9ecdfe3fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:34.207358 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54326fa-7861-46cd-87f4-42a9ecdfe3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:34.207653 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54326fa-7861-46cd-87f4-42a9ecdfe3fe admin admin] PATH_INFO: `/v3/projects/8ce27f4a01c7408aa8b1deb1b01f489d` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:34.209170 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54326fa-7861-46cd-87f4-42a9ecdfe3fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ce27f4a01c7408aa8b1deb1b01f489d)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:34.211214 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54326fa-7861-46cd-87f4-42a9ecdfe3fe admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:34.222132 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0124276-b7d3-4931-b429-a9bae953fbb6 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:34.228793 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d21e7e-4663-4900-ab77-19659bc65f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:34.229628 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d21e7e-4663-4900-ab77-19659bc65f25 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:34.230592 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d21e7e-4663-4900-ab77-19659bc65f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:34.230592 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d21e7e-4663-4900-ab77-19659bc65f25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:34.231057 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d21e7e-4663-4900-ab77-19659bc65f25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:34.233151 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d21e7e-4663-4900-ab77-19659bc65f25 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:34.243131 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:34.265260 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 880/1777] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:34 2022] POST /identity/v3/projects => generated 387 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:34.270286 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d21e7e-4663-4900-ab77-19659bc65f25 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:34.271306 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a54326fa-7861-46cd-87f4-42a9ecdfe3fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ce27f4a01c7408aa8b1deb1b01f489d', 'request_id': 'req-a54326fa-7861-46cd-87f4-42a9ecdfe3fe'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:34.280293 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a54326fa-7861-46cd-87f4-42a9ecdfe3fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ce27f4a01c7408aa8b1deb1b01f489d', 'request_id': 'req-a54326fa-7861-46cd-87f4-42a9ecdfe3fe'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:34.290178 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a54326fa-7861-46cd-87f4-42a9ecdfe3fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ce27f4a01c7408aa8b1deb1b01f489d', 'request_id': 'req-a54326fa-7861-46cd-87f4-42a9ecdfe3fe'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:34.302154 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a54326fa-7861-46cd-87f4-42a9ecdfe3fe admin admin] The token cache is being invalidate because project 8ce27f4a01c7408aa8b1deb1b01f489d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:34.302449 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a54326fa-7861-46cd-87f4-42a9ecdfe3fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:34.303120 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a54326fa-7861-46cd-87f4-42a9ecdfe3fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:34.307959 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 898/1778] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:34 2022] DELETE /identity/v3/projects/8ce27f4a01c7408aa8b1deb1b01f489d => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:34.310081 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54326fa-7861-46cd-87f4-42a9ecdfe3fe admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:34.311508 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a54326fa-7861-46cd-87f4-42a9ecdfe3fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:34.344305 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:34.348609 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d21e7e-4663-4900-ab77-19659bc65f25 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:34.355292 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86460a14-573f-432c-b8c9-510a0fc07a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:34.356011 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86460a14-573f-432c-b8c9-510a0fc07a57 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:34.356288 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86460a14-573f-432c-b8c9-510a0fc07a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:34.356562 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86460a14-573f-432c-b8c9-510a0fc07a57 admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:34.357304 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86460a14-573f-432c-b8c9-510a0fc07a57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:34.359311 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86460a14-573f-432c-b8c9-510a0fc07a57 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:34.385200 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 881/1779] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:34 2022] POST /identity/v3/users => generated 300 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:16:34.386937 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86460a14-573f-432c-b8c9-510a0fc07a57 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:34.389684 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-86460a14-573f-432c-b8c9-510a0fc07a57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:34.399446 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54326fa-7861-46cd-87f4-42a9ecdfe3fe admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:34.403164 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86460a14-573f-432c-b8c9-510a0fc07a57 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:34.406077 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8c3f19-7b57-41fb-8c15-5dda8fe9e67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:34.407077 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c3f19-7b57-41fb-8c15-5dda8fe9e67a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:34.407618 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c3f19-7b57-41fb-8c15-5dda8fe9e67a admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:34.407713 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8c3f19-7b57-41fb-8c15-5dda8fe9e67a admin admin] PATH_INFO: `/v3/users/30efb14fb6a64f8495fe3c2999a321c9` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:34.408549 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b2ae8e-fb21-45dc-84ac-af241453767f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:34.408877 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8c3f19-7b57-41fb-8c15-5dda8fe9e67a admin admin] RBAC: Authorizing `identity:delete_user(user_id=30efb14fb6a64f8495fe3c2999a321c9)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:34.409369 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b2ae8e-fb21-45dc-84ac-af241453767f admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:34.409715 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b2ae8e-fb21-45dc-84ac-af241453767f admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:34.410050 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b2ae8e-fb21-45dc-84ac-af241453767f admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:34.410846 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b2ae8e-fb21-45dc-84ac-af241453767f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:34.410947 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8c3f19-7b57-41fb-8c15-5dda8fe9e67a admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:34.412545 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b2ae8e-fb21-45dc-84ac-af241453767f admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:34.423610 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 882/1780] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:34.427769 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b2ae8e-fb21-45dc-84ac-af241453767f admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:34.439349 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b2ae8e-fb21-45dc-84ac-af241453767f admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:34.444456 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4843ca32-dc4d-419c-8003-adc0999fc110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:34.445420 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843ca32-dc4d-419c-8003-adc0999fc110 admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:34.445727 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843ca32-dc4d-419c-8003-adc0999fc110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:34.446013 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843ca32-dc4d-419c-8003-adc0999fc110 admin admin] PATH_INFO: `/v3/projects/aeb841462f1d4827a72be8f9c6424c94/users/52383732e18d4cde9b04b5d07c37f257/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:34.461455 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-4a8c3f19-7b57-41fb-8c15-5dda8fe9e67a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30efb14fb6a64f8495fe3c2999a321c9'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:34.461740 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4843ca32-dc4d-419c-8003-adc0999fc110 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aeb841462f1d4827a72be8f9c6424c94, user_id=52383732e18d4cde9b04b5d07c37f257, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:34.464811 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4843ca32-dc4d-419c-8003-adc0999fc110 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:34.469992 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-4a8c3f19-7b57-41fb-8c15-5dda8fe9e67a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30efb14fb6a64f8495fe3c2999a321c9'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:34.470752 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-4a8c3f19-7b57-41fb-8c15-5dda8fe9e67a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30efb14fb6a64f8495fe3c2999a321c9'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:34.471438 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-4a8c3f19-7b57-41fb-8c15-5dda8fe9e67a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30efb14fb6a64f8495fe3c2999a321c9'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:34.479193 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 883/1781] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:34 2022] PUT /identity/v3/projects/aeb841462f1d4827a72be8f9c6424c94/users/52383732e18d4cde9b04b5d07c37f257/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:16:34.479602 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-4a8c3f19-7b57-41fb-8c15-5dda8fe9e67a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30efb14fb6a64f8495fe3c2999a321c9'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:34.483218 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4843ca32-dc4d-419c-8003-adc0999fc110 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:34.484891 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-4843ca32-dc4d-419c-8003-adc0999fc110 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:34.498878 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 899/1782] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:16:34 2022] DELETE /identity/v3/users/30efb14fb6a64f8495fe3c2999a321c9 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:34.557105 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4843ca32-dc4d-419c-8003-adc0999fc110 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:34.618558 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aac048-8b45-408a-88fc-de9873dfe447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:34.619258 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aac048-8b45-408a-88fc-de9873dfe447 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:34.619687 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aac048-8b45-408a-88fc-de9873dfe447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:34.619968 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aac048-8b45-408a-88fc-de9873dfe447 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:34.620723 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aac048-8b45-408a-88fc-de9873dfe447 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:34.622380 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aac048-8b45-408a-88fc-de9873dfe447 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:34.633978 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 884/1783] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:34 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:16:34.638871 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8c3f19-7b57-41fb-8c15-5dda8fe9e67a admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:34.651794 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8c3f19-7b57-41fb-8c15-5dda8fe9e67a admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:34.657330 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010b0bd-a8f8-4358-93e1-f257f38ac29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:34.657980 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010b0bd-a8f8-4358-93e1-f257f38ac29c admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:34.658276 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010b0bd-a8f8-4358-93e1-f257f38ac29c admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:34.658610 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010b0bd-a8f8-4358-93e1-f257f38ac29c admin admin] PATH_INFO: `/v3/projects/aeb841462f1d4827a72be8f9c6424c94/users/52383732e18d4cde9b04b5d07c37f257/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:34.660880 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2010b0bd-a8f8-4358-93e1-f257f38ac29c admin admin] RBAC: Authorizing `identity:create_grant(project_id=aeb841462f1d4827a72be8f9c6424c94, user_id=52383732e18d4cde9b04b5d07c37f257, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:34.663639 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2010b0bd-a8f8-4358-93e1-f257f38ac29c admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:34.675706 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 900/1784] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:34 2022] PUT /identity/v3/projects/aeb841462f1d4827a72be8f9c6424c94/users/52383732e18d4cde9b04b5d07c37f257/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:34.687734 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03251176-c545-4c10-a725-dc58ec640650 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:34.688023 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03251176-c545-4c10-a725-dc58ec640650 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:34.688300 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03251176-c545-4c10-a725-dc58ec640650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:34.725159 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-03251176-c545-4c10-a725-dc58ec640650 None None] MFA Rules not processed for user `52383732e18d4cde9b04b5d07c37f257`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:34.726409 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-03251176-c545-4c10-a725-dc58ec640650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:34.779636 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:34.790031 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010b0bd-a8f8-4358-93e1-f257f38ac29c admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:34.792734 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-2010b0bd-a8f8-4358-93e1-f257f38ac29c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:34.812846 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 885/1785] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:34 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:34.819632 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=81,keystone:DELETE=16,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:34.830120 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03251176-c545-4c10-a725-dc58ec640650 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:34.831860 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-03251176-c545-4c10-a725-dc58ec640650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:34.870026 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010b0bd-a8f8-4358-93e1-f257f38ac29c admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:34.875961 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1dd843-9650-402b-8684-dc438ccd2c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:34.876849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1dd843-9650-402b-8684-dc438ccd2c79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:34.877190 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1dd843-9650-402b-8684-dc438ccd2c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:34.877476 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1dd843-9650-402b-8684-dc438ccd2c79 admin admin] PATH_INFO: `/v3/projects/004b92123a9c4dd5af45d06b16bd03b9` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:34.878635 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1dd843-9650-402b-8684-dc438ccd2c79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=004b92123a9c4dd5af45d06b16bd03b9)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:34.880864 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1dd843-9650-402b-8684-dc438ccd2c79 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:34.905211 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03251176-c545-4c10-a725-dc58ec640650 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:34.908942 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81aba06-a249-4e30-93fa-f6341dd697cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:34.909774 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81aba06-a249-4e30-93fa-f6341dd697cc service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:34.910157 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81aba06-a249-4e30-93fa-f6341dd697cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:34.910533 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81aba06-a249-4e30-93fa-f6341dd697cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:34.921422 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81aba06-a249-4e30-93fa-f6341dd697cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:34.923195 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81aba06-a249-4e30-93fa-f6341dd697cc service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:34.934498 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 886/1786] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:34 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:16:34.948629 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ca1dd843-9650-402b-8684-dc438ccd2c79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '004b92123a9c4dd5af45d06b16bd03b9', 'request_id': 'req-ca1dd843-9650-402b-8684-dc438ccd2c79'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:34.966107 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ca1dd843-9650-402b-8684-dc438ccd2c79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '004b92123a9c4dd5af45d06b16bd03b9', 'request_id': 'req-ca1dd843-9650-402b-8684-dc438ccd2c79'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:34.972495 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81aba06-a249-4e30-93fa-f6341dd697cc service cinder] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:34.975453 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ca1dd843-9650-402b-8684-dc438ccd2c79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '004b92123a9c4dd5af45d06b16bd03b9', 'request_id': 'req-ca1dd843-9650-402b-8684-dc438ccd2c79'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:34.985041 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81aba06-a249-4e30-93fa-f6341dd697cc service cinder] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:34.988623 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ca1dd843-9650-402b-8684-dc438ccd2c79 admin admin] The token cache is being invalidate because project 004b92123a9c4dd5af45d06b16bd03b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:34.989244 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ca1dd843-9650-402b-8684-dc438ccd2c79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:34.989977 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ca1dd843-9650-402b-8684-dc438ccd2c79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:34.992817 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 901/1787] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:34 2022] DELETE /identity/v3/projects/004b92123a9c4dd5af45d06b16bd03b9 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:34.999665 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d276fe29-adac-4d9d-a136-070d8369e289 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:35.000324 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d276fe29-adac-4d9d-a136-070d8369e289 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:35.000324 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d276fe29-adac-4d9d-a136-070d8369e289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:35.040455 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-d276fe29-adac-4d9d-a136-070d8369e289 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:35.040455 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d276fe29-adac-4d9d-a136-070d8369e289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:35.084422 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f1199-06ff-4a4e-b2ee-d4d4474b27ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:35.085313 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f1199-06ff-4a4e-b2ee-d4d4474b27ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:35.085901 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f1199-06ff-4a4e-b2ee-d4d4474b27ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:35.086186 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f1199-06ff-4a4e-b2ee-d4d4474b27ad admin admin] PATH_INFO: `/v3/users/d6091238fa5b471481c5115703e33122` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:35.087578 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815f1199-06ff-4a4e-b2ee-d4d4474b27ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6091238fa5b471481c5115703e33122)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:35.090641 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815f1199-06ff-4a4e-b2ee-d4d4474b27ad admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:35.121423 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 902/1788] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:16:35.130969 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cf8056-9ee6-449f-99ce-81a094c374b8 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:35.131254 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cf8056-9ee6-449f-99ce-81a094c374b8 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:35.131531 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cf8056-9ee6-449f-99ce-81a094c374b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:35.141209 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-815f1199-06ff-4a4e-b2ee-d4d4474b27ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6091238fa5b471481c5115703e33122'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:35.159356 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-68cf8056-9ee6-449f-99ce-81a094c374b8 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:35.159574 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-815f1199-06ff-4a4e-b2ee-d4d4474b27ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6091238fa5b471481c5115703e33122'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:35.160346 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-815f1199-06ff-4a4e-b2ee-d4d4474b27ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6091238fa5b471481c5115703e33122'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:35.160613 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-68cf8056-9ee6-449f-99ce-81a094c374b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:35.161036 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-815f1199-06ff-4a4e-b2ee-d4d4474b27ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6091238fa5b471481c5115703e33122'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:35.172558 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 903/1789] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:35.172863 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-815f1199-06ff-4a4e-b2ee-d4d4474b27ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6091238fa5b471481c5115703e33122'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:35.173850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cf8056-9ee6-449f-99ce-81a094c374b8 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:35.187160 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cf8056-9ee6-449f-99ce-81a094c374b8 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:35.193402 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fea1dd-b03c-4ecf-a79c-df2dde533556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:35.194510 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fea1dd-b03c-4ecf-a79c-df2dde533556 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:35.195698 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fea1dd-b03c-4ecf-a79c-df2dde533556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:35.196096 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fea1dd-b03c-4ecf-a79c-df2dde533556 admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:35.197724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fea1dd-b03c-4ecf-a79c-df2dde533556 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:16:35.197724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fea1dd-b03c-4ecf-a79c-df2dde533556 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:35.199417 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fea1dd-b03c-4ecf-a79c-df2dde533556 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:35.203808 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 887/1790] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:16:34 2022] DELETE /identity/v3/users/d6091238fa5b471481c5115703e33122 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:16:35.209750 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c309f81-13d5-42d2-b339-cb0279712ac1 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:35.210609 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c309f81-13d5-42d2-b339-cb0279712ac1 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:35.210966 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c309f81-13d5-42d2-b339-cb0279712ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:35.217039 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 904/1791] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:16:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:16:35.219726 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fea1dd-b03c-4ecf-a79c-df2dde533556 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:35.222244 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:35.232785 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fea1dd-b03c-4ecf-a79c-df2dde533556 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:35.248875 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-2c309f81-13d5-42d2-b339-cb0279712ac1 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:35.250884 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-2c309f81-13d5-42d2-b339-cb0279712ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:35.271371 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:35.313752 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3490eeb8-470f-4dcc-b6cc-82ea9f332e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:35.314435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490eeb8-470f-4dcc-b6cc-82ea9f332e2c admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:35.314868 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490eeb8-470f-4dcc-b6cc-82ea9f332e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:35.315151 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490eeb8-470f-4dcc-b6cc-82ea9f332e2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:35.315856 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3490eeb8-470f-4dcc-b6cc-82ea9f332e2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:35.317824 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3490eeb8-470f-4dcc-b6cc-82ea9f332e2c admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:35.334725 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 888/1792] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:35.342355 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c309f81-13d5-42d2-b339-cb0279712ac1 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:35.345493 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-2c309f81-13d5-42d2-b339-cb0279712ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:35.363470 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 905/1793] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:35 2022] POST /identity/v3/projects => generated 373 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:16:35.367595 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3490eeb8-470f-4dcc-b6cc-82ea9f332e2c admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:35.413746 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c309f81-13d5-42d2-b339-cb0279712ac1 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:35.451900 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3490eeb8-470f-4dcc-b6cc-82ea9f332e2c admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:35.457482 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18792ad8-4e75-465f-9b37-c73bf4d72e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:35.458285 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18792ad8-4e75-465f-9b37-c73bf4d72e8d admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:35.458660 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18792ad8-4e75-465f-9b37-c73bf4d72e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:35.459054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18792ad8-4e75-465f-9b37-c73bf4d72e8d admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:35.459974 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18792ad8-4e75-465f-9b37-c73bf4d72e8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:35.462086 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18792ad8-4e75-465f-9b37-c73bf4d72e8d admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:35.490338 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 906/1794] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:35 2022] POST /identity/v3/users => generated 287 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:16:35.495617 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18792ad8-4e75-465f-9b37-c73bf4d72e8d admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:35.505700 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6afabb4-de05-4583-98e7-d8e9377024a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:35.507884 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6afabb4-de05-4583-98e7-d8e9377024a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:35.507884 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6afabb4-de05-4583-98e7-d8e9377024a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:35.508230 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6afabb4-de05-4583-98e7-d8e9377024a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:35.514856 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18792ad8-4e75-465f-9b37-c73bf4d72e8d admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:35.521568 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e6490-40b6-4e19-87ed-8e0e6cf688c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:35.521755 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6afabb4-de05-4583-98e7-d8e9377024a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:35.534290 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e6490-40b6-4e19-87ed-8e0e6cf688c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:35.534290 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e6490-40b6-4e19-87ed-8e0e6cf688c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:35.534290 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e6490-40b6-4e19-87ed-8e0e6cf688c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:35.534290 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811e6490-40b6-4e19-87ed-8e0e6cf688c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:35.534851 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6afabb4-de05-4583-98e7-d8e9377024a5 service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:35.534949 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811e6490-40b6-4e19-87ed-8e0e6cf688c1 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:35.538080 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 889/1795] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:35.538589 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 907/1796] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:35 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:35.542538 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6afabb4-de05-4583-98e7-d8e9377024a5 service neutron] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:35.552566 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6afabb4-de05-4583-98e7-d8e9377024a5 service neutron] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:35.558636 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d65fc6-78da-462f-8b4b-143851969fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:35.560177 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d65fc6-78da-462f-8b4b-143851969fcc admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:35.560609 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d65fc6-78da-462f-8b4b-143851969fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:35.560889 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d65fc6-78da-462f-8b4b-143851969fcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:35.561957 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d65fc6-78da-462f-8b4b-143851969fcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:35.563811 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d65fc6-78da-462f-8b4b-143851969fcc admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:35.578880 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 890/1797] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:35.580166 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e6490-40b6-4e19-87ed-8e0e6cf688c1 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:35.591909 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e6490-40b6-4e19-87ed-8e0e6cf688c1 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:35.596526 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1921dfc-6e70-4ed2-aa4d-c5abeaf92d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:35.597484 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1921dfc-6e70-4ed2-aa4d-c5abeaf92d55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:35.597761 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1921dfc-6e70-4ed2-aa4d-c5abeaf92d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:35.598032 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1921dfc-6e70-4ed2-aa4d-c5abeaf92d55 admin admin] PATH_INFO: `/v3/projects/7fb32a0153c1491bb10c2e798f1fcf46/users/7b98f3b3cf174d678af5d8c9f2949018/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:35.614628 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1921dfc-6e70-4ed2-aa4d-c5abeaf92d55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fb32a0153c1491bb10c2e798f1fcf46, user_id=7b98f3b3cf174d678af5d8c9f2949018, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:35.617360 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1921dfc-6e70-4ed2-aa4d-c5abeaf92d55 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:35.640897 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 908/1798] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:35 2022] PUT /identity/v3/projects/7fb32a0153c1491bb10c2e798f1fcf46/users/7b98f3b3cf174d678af5d8c9f2949018/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:35.646307 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6473376-3c32-4865-8a90-4fb84322f739 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:35.646664 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6473376-3c32-4865-8a90-4fb84322f739 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:35.646842 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6473376-3c32-4865-8a90-4fb84322f739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:35.666242 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-e6473376-3c32-4865-8a90-4fb84322f739 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:35.667513 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-e6473376-3c32-4865-8a90-4fb84322f739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:35.715423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:35.732980 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 891/1799] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:35.744359 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1921dfc-6e70-4ed2-aa4d-c5abeaf92d55 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:35.746279 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-e1921dfc-6e70-4ed2-aa4d-c5abeaf92d55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:35.872099 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1921dfc-6e70-4ed2-aa4d-c5abeaf92d55 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:35.882451 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e57d2-d8e8-40b8-aab6-816293d50ca8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:35.883245 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e57d2-d8e8-40b8-aab6-816293d50ca8 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:35.883595 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e57d2-d8e8-40b8-aab6-816293d50ca8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:35.883935 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e57d2-d8e8-40b8-aab6-816293d50ca8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:35.898787 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6e57d2-d8e8-40b8-aab6-816293d50ca8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:35.900975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6e57d2-d8e8-40b8-aab6-816293d50ca8 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:35.914820 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 909/1800] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:16:36.395943 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6473376-3c32-4865-8a90-4fb84322f739 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:36.397891 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-e6473376-3c32-4865-8a90-4fb84322f739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:36.411502 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6473376-3c32-4865-8a90-4fb84322f739 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:36.416402 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d9ad1-8bea-4303-bcb7-686e15b8fed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:36.417354 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d9ad1-8bea-4303-bcb7-686e15b8fed1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:36.417637 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d9ad1-8bea-4303-bcb7-686e15b8fed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:36.418059 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d9ad1-8bea-4303-bcb7-686e15b8fed1 admin admin] PATH_INFO: `/v3/projects/a5f6d7a881ba4106b694e00702c8a502` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:36.419305 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860d9ad1-8bea-4303-bcb7-686e15b8fed1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5f6d7a881ba4106b694e00702c8a502)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:36.421286 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860d9ad1-8bea-4303-bcb7-686e15b8fed1 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:36.493937 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-860d9ad1-8bea-4303-bcb7-686e15b8fed1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5f6d7a881ba4106b694e00702c8a502', 'request_id': 'req-860d9ad1-8bea-4303-bcb7-686e15b8fed1'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:36.501885 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-860d9ad1-8bea-4303-bcb7-686e15b8fed1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5f6d7a881ba4106b694e00702c8a502', 'request_id': 'req-860d9ad1-8bea-4303-bcb7-686e15b8fed1'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:36.510856 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-860d9ad1-8bea-4303-bcb7-686e15b8fed1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5f6d7a881ba4106b694e00702c8a502', 'request_id': 'req-860d9ad1-8bea-4303-bcb7-686e15b8fed1'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:36.524907 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-860d9ad1-8bea-4303-bcb7-686e15b8fed1 admin admin] The token cache is being invalidate because project a5f6d7a881ba4106b694e00702c8a502 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:36.525896 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-860d9ad1-8bea-4303-bcb7-686e15b8fed1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:36.526604 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-860d9ad1-8bea-4303-bcb7-686e15b8fed1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:36.528248 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 892/1801] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:36 2022] DELETE /identity/v3/projects/a5f6d7a881ba4106b694e00702c8a502 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:36.533492 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e57d2-d8e8-40b8-aab6-816293d50ca8 service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:36.549574 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e57d2-d8e8-40b8-aab6-816293d50ca8 service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:36.562732 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:36.655577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc2b7e-308c-425e-82cb-ca1997c4cc48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:36.657425 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc2b7e-308c-425e-82cb-ca1997c4cc48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:36.657898 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc2b7e-308c-425e-82cb-ca1997c4cc48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:36.658404 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc2b7e-308c-425e-82cb-ca1997c4cc48 admin admin] PATH_INFO: `/v3/users/52383732e18d4cde9b04b5d07c37f257` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:36.660638 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcc2b7e-308c-425e-82cb-ca1997c4cc48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52383732e18d4cde9b04b5d07c37f257)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:36.663919 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcc2b7e-308c-425e-82cb-ca1997c4cc48 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:36.741397 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-efcc2b7e-308c-425e-82cb-ca1997c4cc48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52383732e18d4cde9b04b5d07c37f257'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:36.750854 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-efcc2b7e-308c-425e-82cb-ca1997c4cc48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52383732e18d4cde9b04b5d07c37f257'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:36.752810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-efcc2b7e-308c-425e-82cb-ca1997c4cc48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52383732e18d4cde9b04b5d07c37f257'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:36.754023 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-efcc2b7e-308c-425e-82cb-ca1997c4cc48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52383732e18d4cde9b04b5d07c37f257'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:36.765696 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-efcc2b7e-308c-425e-82cb-ca1997c4cc48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52383732e18d4cde9b04b5d07c37f257'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:36.789929 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 910/1802] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:16:36 2022] DELETE /identity/v3/users/52383732e18d4cde9b04b5d07c37f257 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:37.697521 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d9ad1-8bea-4303-bcb7-686e15b8fed1 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:37.700310 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-860d9ad1-8bea-4303-bcb7-686e15b8fed1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:37.725589 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d9ad1-8bea-4303-bcb7-686e15b8fed1 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:37.866791 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5084545a-5f91-41c4-b53a-4ea0c20024ff None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:37.867338 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5084545a-5f91-41c4-b53a-4ea0c20024ff None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:37.868210 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5084545a-5f91-41c4-b53a-4ea0c20024ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:37.893908 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55abbb36-a7b1-420a-9920-0eae6da1357a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:37.895031 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55abbb36-a7b1-420a-9920-0eae6da1357a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:37.895486 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55abbb36-a7b1-420a-9920-0eae6da1357a admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:37.895921 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55abbb36-a7b1-420a-9920-0eae6da1357a admin admin] PATH_INFO: `/v3/projects/aeb841462f1d4827a72be8f9c6424c94` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:37.898179 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55abbb36-a7b1-420a-9920-0eae6da1357a admin admin] RBAC: Authorizing `identity:delete_project(project_id=aeb841462f1d4827a72be8f9c6424c94)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:37.900827 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55abbb36-a7b1-420a-9920-0eae6da1357a admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:37.950148 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-5084545a-5f91-41c4-b53a-4ea0c20024ff None None] MFA Rules not processed for user `7b98f3b3cf174d678af5d8c9f2949018`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:37.951394 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-5084545a-5f91-41c4-b53a-4ea0c20024ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:38.036523 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-55abbb36-a7b1-420a-9920-0eae6da1357a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aeb841462f1d4827a72be8f9c6424c94', 'request_id': 'req-55abbb36-a7b1-420a-9920-0eae6da1357a'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:38.047873 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-55abbb36-a7b1-420a-9920-0eae6da1357a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aeb841462f1d4827a72be8f9c6424c94', 'request_id': 'req-55abbb36-a7b1-420a-9920-0eae6da1357a'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:38.063137 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-55abbb36-a7b1-420a-9920-0eae6da1357a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aeb841462f1d4827a72be8f9c6424c94', 'request_id': 'req-55abbb36-a7b1-420a-9920-0eae6da1357a'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:38.089179 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-55abbb36-a7b1-420a-9920-0eae6da1357a admin admin] The token cache is being invalidate because project aeb841462f1d4827a72be8f9c6424c94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:38.091840 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-55abbb36-a7b1-420a-9920-0eae6da1357a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:38.094546 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-55abbb36-a7b1-420a-9920-0eae6da1357a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:38.099555 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 893/1803] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:37 2022] DELETE /identity/v3/projects/aeb841462f1d4827a72be8f9c6424c94 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:38.106628 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22756c-b0ea-4db6-b1ce-fe08fffe8258 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:38.107433 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22756c-b0ea-4db6-b1ce-fe08fffe8258 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:38.108404 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22756c-b0ea-4db6-b1ce-fe08fffe8258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:38.136332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:38.148106 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 911/1804] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:38.154282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5084545a-5f91-41c4-b53a-4ea0c20024ff None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:38.154869 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-9e22756c-b0ea-4db6-b1ce-fe08fffe8258 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:38.157056 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-5084545a-5f91-41c4-b53a-4ea0c20024ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:38.157273 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-9e22756c-b0ea-4db6-b1ce-fe08fffe8258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:38.179334 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5084545a-5f91-41c4-b53a-4ea0c20024ff None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:38.202049 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:38.235788 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 894/1805] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:38.245934 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f2044-690a-460d-930a-1b1a92427e33 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:38.246371 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f2044-690a-460d-930a-1b1a92427e33 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:38.246749 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f2044-690a-460d-930a-1b1a92427e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:38.281654 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-c72f2044-690a-460d-930a-1b1a92427e33 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:38.284424 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-c72f2044-690a-460d-930a-1b1a92427e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:38.297647 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 895/1806] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:38.301766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f2044-690a-460d-930a-1b1a92427e33 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:38.305373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28551208-998e-4984-af3f-ff6610764f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:38.306425 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28551208-998e-4984-af3f-ff6610764f29 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:38.307312 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28551208-998e-4984-af3f-ff6610764f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:38.309004 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28551208-998e-4984-af3f-ff6610764f29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:38.310712 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28551208-998e-4984-af3f-ff6610764f29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:38.314639 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28551208-998e-4984-af3f-ff6610764f29 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:38.315056 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f2044-690a-460d-930a-1b1a92427e33 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:38.328753 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec8884-993b-4a0a-8a91-4bafc10540dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:38.331402 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec8884-993b-4a0a-8a91-4bafc10540dd admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:38.332760 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec8884-993b-4a0a-8a91-4bafc10540dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:38.335653 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec8884-993b-4a0a-8a91-4bafc10540dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:38.336980 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec8884-993b-4a0a-8a91-4bafc10540dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:16:38.339732 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec8884-993b-4a0a-8a91-4bafc10540dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:38.346148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec8884-993b-4a0a-8a91-4bafc10540dd admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:38.372280 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 896/1807] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:16:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:16:38.374718 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec8884-993b-4a0a-8a91-4bafc10540dd admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:38.385475 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 912/1808] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:38 2022] POST /identity/v3/projects => generated 371 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:16:38.387413 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec8884-993b-4a0a-8a91-4bafc10540dd admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:38.389946 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28551208-998e-4984-af3f-ff6610764f29 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:38.482166 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91ef14-338c-496e-9078-d333c11a529f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:38.483153 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91ef14-338c-496e-9078-d333c11a529f admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:38.483620 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91ef14-338c-496e-9078-d333c11a529f admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:38.483896 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91ef14-338c-496e-9078-d333c11a529f admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:38.484617 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb91ef14-338c-496e-9078-d333c11a529f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:38.487613 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb91ef14-338c-496e-9078-d333c11a529f admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:38.488685 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28551208-998e-4984-af3f-ff6610764f29 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:38.497159 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca79d6-5203-4e31-904b-bc31964193b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:38.498017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca79d6-5203-4e31-904b-bc31964193b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:38.498418 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca79d6-5203-4e31-904b-bc31964193b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:38.498946 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca79d6-5203-4e31-904b-bc31964193b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:38.500066 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ca79d6-5203-4e31-904b-bc31964193b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:38.502874 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ca79d6-5203-4e31-904b-bc31964193b0 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:38.529631 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 897/1809] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:38 2022] POST /identity/v3/projects => generated 381 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:38.530754 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91ef14-338c-496e-9078-d333c11a529f admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:38.536474 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 913/1810] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:38 2022] POST /identity/v3/users => generated 292 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:16:38.537743 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca79d6-5203-4e31-904b-bc31964193b0 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:38.551258 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca79d6-5203-4e31-904b-bc31964193b0 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:38.649399 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91ef14-338c-496e-9078-d333c11a529f admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:38.655368 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397396a0-1f65-4e7f-9218-91bf4660c7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:38.656072 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397396a0-1f65-4e7f-9218-91bf4660c7c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:38.656353 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397396a0-1f65-4e7f-9218-91bf4660c7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:38.656628 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397396a0-1f65-4e7f-9218-91bf4660c7c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:38.657397 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397396a0-1f65-4e7f-9218-91bf4660c7c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:38.659551 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397396a0-1f65-4e7f-9218-91bf4660c7c1 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:38.662741 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b5c8f-dbf4-49e1-8e8f-b30d9cc3f69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:38.663444 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b5c8f-dbf4-49e1-8e8f-b30d9cc3f69d admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:38.663728 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b5c8f-dbf4-49e1-8e8f-b30d9cc3f69d admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:38.664004 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b5c8f-dbf4-49e1-8e8f-b30d9cc3f69d admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:38.664823 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b5c8f-dbf4-49e1-8e8f-b30d9cc3f69d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:38.667839 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b5c8f-dbf4-49e1-8e8f-b30d9cc3f69d admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:38.684512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:38.693333 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 898/1811] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:38 2022] POST /identity/v3/users => generated 291 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:38.694242 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 914/1812] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:38 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:38.702405 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b5c8f-dbf4-49e1-8e8f-b30d9cc3f69d admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:38.702838 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397396a0-1f65-4e7f-9218-91bf4660c7c1 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:38.722175 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397396a0-1f65-4e7f-9218-91bf4660c7c1 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:38.729004 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b5c8f-dbf4-49e1-8e8f-b30d9cc3f69d admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:38.734008 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e4993-3a30-4219-af8f-4ea0090418c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:38.735059 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e4993-3a30-4219-af8f-4ea0090418c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:38.735405 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e4993-3a30-4219-af8f-4ea0090418c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:38.736559 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e4993-3a30-4219-af8f-4ea0090418c9 admin admin] PATH_INFO: `/v3/projects/f044bbb8bb2d4af9ae27d3b849e3a369/users/2bbf85747f624ab6bfb4b4eec90c3ca8/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:38.738039 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee60ee8c-8db9-4dc5-a0a4-31958f6ecb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:38.738727 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60ee8c-8db9-4dc5-a0a4-31958f6ecb7c admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:38.739006 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60ee8c-8db9-4dc5-a0a4-31958f6ecb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:38.739275 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60ee8c-8db9-4dc5-a0a4-31958f6ecb7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:38.740372 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee60ee8c-8db9-4dc5-a0a4-31958f6ecb7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:38.742075 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee60ee8c-8db9-4dc5-a0a4-31958f6ecb7c admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:38.761483 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8e4993-3a30-4219-af8f-4ea0090418c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f044bbb8bb2d4af9ae27d3b849e3a369, user_id=2bbf85747f624ab6bfb4b4eec90c3ca8, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:38.764858 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8e4993-3a30-4219-af8f-4ea0090418c9 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:38.784947 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 899/1813] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:38 2022] PUT /identity/v3/projects/f044bbb8bb2d4af9ae27d3b849e3a369/users/2bbf85747f624ab6bfb4b4eec90c3ca8/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:38.788455 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e4993-3a30-4219-af8f-4ea0090418c9 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:38.799384 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 915/1814] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:38 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:38.811403 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee60ee8c-8db9-4dc5-a0a4-31958f6ecb7c admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:38.842077 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee60ee8c-8db9-4dc5-a0a4-31958f6ecb7c admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:38.888030 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:38.908530 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e4993-3a30-4219-af8f-4ea0090418c9 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:38.921109 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948b3ef-f714-48b2-afe3-b62917416fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:38.921810 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948b3ef-f714-48b2-afe3-b62917416fe6 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:38.922094 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948b3ef-f714-48b2-afe3-b62917416fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:38.922371 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948b3ef-f714-48b2-afe3-b62917416fe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:38.923627 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a948b3ef-f714-48b2-afe3-b62917416fe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:38.925414 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a948b3ef-f714-48b2-afe3-b62917416fe6 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:38.939790 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 900/1815] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:38 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:16:38.944059 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948b3ef-f714-48b2-afe3-b62917416fe6 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:38.955505 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948b3ef-f714-48b2-afe3-b62917416fe6 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:38.964600 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48bd558-b7da-4f87-b6e1-0e33719a9689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:38.965621 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48bd558-b7da-4f87-b6e1-0e33719a9689 admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:38.965896 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48bd558-b7da-4f87-b6e1-0e33719a9689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:38.966162 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48bd558-b7da-4f87-b6e1-0e33719a9689 admin admin] PATH_INFO: `/v3/projects/f044bbb8bb2d4af9ae27d3b849e3a369/users/2bbf85747f624ab6bfb4b4eec90c3ca8/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:38.969378 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48bd558-b7da-4f87-b6e1-0e33719a9689 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f044bbb8bb2d4af9ae27d3b849e3a369, user_id=2bbf85747f624ab6bfb4b4eec90c3ca8, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:38.972591 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48bd558-b7da-4f87-b6e1-0e33719a9689 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:39.001735 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 901/1816] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:38 2022] PUT /identity/v3/projects/f044bbb8bb2d4af9ae27d3b849e3a369/users/2bbf85747f624ab6bfb4b4eec90c3ca8/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:16:39.039144 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9ce22-aab1-4ef2-adf4-545573b06824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:39.039842 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9ce22-aab1-4ef2-adf4-545573b06824 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:39.040129 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9ce22-aab1-4ef2-adf4-545573b06824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:39.040461 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9ce22-aab1-4ef2-adf4-545573b06824 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:39.041426 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a9ce22-aab1-4ef2-adf4-545573b06824 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:39.043463 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a9ce22-aab1-4ef2-adf4-545573b06824 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:39.056537 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 916/1817] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:38 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:39.062287 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48bd558-b7da-4f87-b6e1-0e33719a9689 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:39.213064 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48bd558-b7da-4f87-b6e1-0e33719a9689 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:39.231165 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4083d2c-3d9d-40b6-80ec-0480bf402931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:39.232184 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4083d2c-3d9d-40b6-80ec-0480bf402931 admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:39.232532 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4083d2c-3d9d-40b6-80ec-0480bf402931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:39.232880 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4083d2c-3d9d-40b6-80ec-0480bf402931 admin admin] PATH_INFO: `/v3/projects/9cf6a289a0834a4a8b02325cd9e00ab6/users/3a9d25cbfa5e4093b29907453a632703/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:39.269267 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4083d2c-3d9d-40b6-80ec-0480bf402931 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cf6a289a0834a4a8b02325cd9e00ab6, user_id=3a9d25cbfa5e4093b29907453a632703, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:39.272336 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4083d2c-3d9d-40b6-80ec-0480bf402931 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:39.296070 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 902/1818] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:39 2022] PUT /identity/v3/projects/9cf6a289a0834a4a8b02325cd9e00ab6/users/3a9d25cbfa5e4093b29907453a632703/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:39.309337 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3811fa7e-a4e6-4d57-8418-6ed226c9fc73 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:39.309828 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3811fa7e-a4e6-4d57-8418-6ed226c9fc73 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:39.310303 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3811fa7e-a4e6-4d57-8418-6ed226c9fc73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:39.357308 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-3811fa7e-a4e6-4d57-8418-6ed226c9fc73 None None] MFA Rules not processed for user `3a9d25cbfa5e4093b29907453a632703`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:39.358856 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-3811fa7e-a4e6-4d57-8418-6ed226c9fc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:39.497768 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 917/1819] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:39.499565 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4083d2c-3d9d-40b6-80ec-0480bf402931 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:39.566608 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:39.598402 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4083d2c-3d9d-40b6-80ec-0480bf402931 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:39.606394 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c186781-3eb1-4fc8-89a5-264e18ccade3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:39.607610 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c186781-3eb1-4fc8-89a5-264e18ccade3 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:39.608077 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c186781-3eb1-4fc8-89a5-264e18ccade3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:39.608468 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c186781-3eb1-4fc8-89a5-264e18ccade3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:39.610525 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c186781-3eb1-4fc8-89a5-264e18ccade3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:39.612778 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c186781-3eb1-4fc8-89a5-264e18ccade3 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:39.667652 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 903/1820] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:39 2022] POST /identity/v3/projects => generated 381 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:39.673308 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3811fa7e-a4e6-4d57-8418-6ed226c9fc73 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:39.689413 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3811fa7e-a4e6-4d57-8418-6ed226c9fc73 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:39.787224 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92065e6d-9ea5-4d97-b58d-112d2d0b19d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:39.788653 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92065e6d-9ea5-4d97-b58d-112d2d0b19d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:39.789617 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92065e6d-9ea5-4d97-b58d-112d2d0b19d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:39.791142 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92065e6d-9ea5-4d97-b58d-112d2d0b19d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:39.792791 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92065e6d-9ea5-4d97-b58d-112d2d0b19d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:39.795934 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92065e6d-9ea5-4d97-b58d-112d2d0b19d2 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:39.840976 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 918/1821] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:39 2022] POST /identity/v3/users => generated 297 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:39.842608 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c186781-3eb1-4fc8-89a5-264e18ccade3 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:39.873149 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c186781-3eb1-4fc8-89a5-264e18ccade3 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:39.881688 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4c71e-b894-4fcc-883d-8b45e210b953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:39.885592 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4c71e-b894-4fcc-883d-8b45e210b953 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:39.886363 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4c71e-b894-4fcc-883d-8b45e210b953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:39.886949 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4c71e-b894-4fcc-883d-8b45e210b953 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:39.888082 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae4c71e-b894-4fcc-883d-8b45e210b953 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:39.896824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae4c71e-b894-4fcc-883d-8b45e210b953 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:39.916477 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 904/1822] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:39 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:39.917868 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92065e6d-9ea5-4d97-b58d-112d2d0b19d2 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:39.940121 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92065e6d-9ea5-4d97-b58d-112d2d0b19d2 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:39.946424 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ff8cd-ea39-4c40-9522-8f3fb810c10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:39.948178 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ff8cd-ea39-4c40-9522-8f3fb810c10c admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:39.948465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ff8cd-ea39-4c40-9522-8f3fb810c10c admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:39.949873 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ff8cd-ea39-4c40-9522-8f3fb810c10c admin admin] PATH_INFO: `/v3/projects/3050c89ed098482c81c0f6b3f0f8b451/users/fda3d835da0d429bbf44a17fbd502f4a/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:39.974923 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9ff8cd-ea39-4c40-9522-8f3fb810c10c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3050c89ed098482c81c0f6b3f0f8b451, user_id=fda3d835da0d429bbf44a17fbd502f4a, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:39.977970 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9ff8cd-ea39-4c40-9522-8f3fb810c10c admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:39.995551 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 919/1823] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:39 2022] PUT /identity/v3/projects/3050c89ed098482c81c0f6b3f0f8b451/users/fda3d835da0d429bbf44a17fbd502f4a/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:39.998797 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4c71e-b894-4fcc-883d-8b45e210b953 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:40.036779 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4c71e-b894-4fcc-883d-8b45e210b953 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:40.167069 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374189d-7cd4-4ffb-b9a2-9fe139c28884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:40.168250 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374189d-7cd4-4ffb-b9a2-9fe139c28884 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:40.169091 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374189d-7cd4-4ffb-b9a2-9fe139c28884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:40.169594 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374189d-7cd4-4ffb-b9a2-9fe139c28884 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:40.170801 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d374189d-7cd4-4ffb-b9a2-9fe139c28884 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:40.173396 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d374189d-7cd4-4ffb-b9a2-9fe139c28884 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:40.194281 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 905/1824] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:39 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:40.194571 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ff8cd-ea39-4c40-9522-8f3fb810c10c admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:40.221346 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ff8cd-ea39-4c40-9522-8f3fb810c10c admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:40.233301 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105c115f-5cae-4f63-a244-3785eb43a5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:40.234230 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c115f-5cae-4f63-a244-3785eb43a5fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:40.234778 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c115f-5cae-4f63-a244-3785eb43a5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:40.235061 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c115f-5cae-4f63-a244-3785eb43a5fc admin admin] PATH_INFO: `/v3/projects/3050c89ed098482c81c0f6b3f0f8b451/users/fda3d835da0d429bbf44a17fbd502f4a/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:40.249884 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105c115f-5cae-4f63-a244-3785eb43a5fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3050c89ed098482c81c0f6b3f0f8b451, user_id=fda3d835da0d429bbf44a17fbd502f4a, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:40.258988 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105c115f-5cae-4f63-a244-3785eb43a5fc admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:40.267289 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:40.284121 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 920/1825] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:40 2022] PUT /identity/v3/projects/3050c89ed098482c81c0f6b3f0f8b451/users/fda3d835da0d429bbf44a17fbd502f4a/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:16:40.298251 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f68ebd-d5a7-41e8-aacd-a10c23582812 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:40.303050 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f68ebd-d5a7-41e8-aacd-a10c23582812 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:40.303816 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f68ebd-d5a7-41e8-aacd-a10c23582812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:40.393832 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-c2f68ebd-d5a7-41e8-aacd-a10c23582812 None None] MFA Rules not processed for user `fda3d835da0d429bbf44a17fbd502f4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:40.399939 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-c2f68ebd-d5a7-41e8-aacd-a10c23582812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:40.419324 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105c115f-5cae-4f63-a244-3785eb43a5fc admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:40.421861 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-105c115f-5cae-4f63-a244-3785eb43a5fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:40.581269 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 906/1826] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:40 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:40.583784 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105c115f-5cae-4f63-a244-3785eb43a5fc admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:40.590177 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f68ebd-d5a7-41e8-aacd-a10c23582812 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:40.592235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-c2f68ebd-d5a7-41e8-aacd-a10c23582812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:40.600433 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed6e25f-fb3e-4417-a25c-e6096a6932cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:40.602283 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6e25f-fb3e-4417-a25c-e6096a6932cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:40.602283 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6e25f-fb3e-4417-a25c-e6096a6932cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:40.602283 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed6e25f-fb3e-4417-a25c-e6096a6932cb admin admin] PATH_INFO: `/v3/users/eb061d0e9a1e490f93c319ec554f7599` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:40.603765 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed6e25f-fb3e-4417-a25c-e6096a6932cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb061d0e9a1e490f93c319ec554f7599)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:40.605715 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed6e25f-fb3e-4417-a25c-e6096a6932cb admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:40.673083 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:40.698530 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f68ebd-d5a7-41e8-aacd-a10c23582812 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:40.703662 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36bf55-e119-44a5-9b32-1f2a4caeba35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:40.704626 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36bf55-e119-44a5-9b32-1f2a4caeba35 service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:40.705167 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36bf55-e119-44a5-9b32-1f2a4caeba35 service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:40.705659 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36bf55-e119-44a5-9b32-1f2a4caeba35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:40.717509 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-5ed6e25f-fb3e-4417-a25c-e6096a6932cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb061d0e9a1e490f93c319ec554f7599'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:40.722255 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f36bf55-e119-44a5-9b32-1f2a4caeba35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:40.737627 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-5ed6e25f-fb3e-4417-a25c-e6096a6932cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb061d0e9a1e490f93c319ec554f7599'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:40.738580 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-5ed6e25f-fb3e-4417-a25c-e6096a6932cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb061d0e9a1e490f93c319ec554f7599'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:40.739771 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-5ed6e25f-fb3e-4417-a25c-e6096a6932cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb061d0e9a1e490f93c319ec554f7599'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:40.751533 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f36bf55-e119-44a5-9b32-1f2a4caeba35 service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:40.766803 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-5ed6e25f-fb3e-4417-a25c-e6096a6932cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb061d0e9a1e490f93c319ec554f7599'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:40.805340 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 921/1827] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:16:40 2022] DELETE /identity/v3/users/eb061d0e9a1e490f93c319ec554f7599 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:40.815578 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 907/1828] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:40 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:40.820085 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b3b37-8c39-459b-a20b-625bcefa0f8b None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:40.820377 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b3b37-8c39-459b-a20b-625bcefa0f8b None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:40.820661 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b3b37-8c39-459b-a20b-625bcefa0f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:40.841833 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-1e4b3b37-8c39-459b-a20b-625bcefa0f8b None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:40.843077 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-1e4b3b37-8c39-459b-a20b-625bcefa0f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:40.925460 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 922/1829] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:40.970110 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36bf55-e119-44a5-9b32-1f2a4caeba35 service cinder] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:40.972618 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b3b37-8c39-459b-a20b-625bcefa0f8b None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:40.973382 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-4f36bf55-e119-44a5-9b32-1f2a4caeba35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:40.974414 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-1e4b3b37-8c39-459b-a20b-625bcefa0f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:41.022226 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:41.097590 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36bf55-e119-44a5-9b32-1f2a4caeba35 service cinder] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:41.124164 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b3b37-8c39-459b-a20b-625bcefa0f8b None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:41.128948 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a5c706-db7e-4186-8ba3-af657d2f22a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:41.129864 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a5c706-db7e-4186-8ba3-af657d2f22a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:41.130033 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a5c706-db7e-4186-8ba3-af657d2f22a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:41.130304 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a5c706-db7e-4186-8ba3-af657d2f22a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:41.132559 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-55a5c706-db7e-4186-8ba3-af657d2f22a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:41.205605 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c04aa-6247-45ef-a968-ede5522e79d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:41.209534 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c04aa-6247-45ef-a968-ede5522e79d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:41.210209 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c04aa-6247-45ef-a968-ede5522e79d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:41.211622 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c04aa-6247-45ef-a968-ede5522e79d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:41.232040 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a5c706-db7e-4186-8ba3-af657d2f22a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:41.234875 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a5c706-db7e-4186-8ba3-af657d2f22a5 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:41.238767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06c04aa-6247-45ef-a968-ede5522e79d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:41.240421 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06c04aa-6247-45ef-a968-ede5522e79d2 service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:41.275806 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 923/1830] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:16:41.287501 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 908/1831] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:42.372650 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a5c706-db7e-4186-8ba3-af657d2f22a5 service cinder] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:42.375560 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-55a5c706-db7e-4186-8ba3-af657d2f22a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:42.403659 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a5c706-db7e-4186-8ba3-af657d2f22a5 service cinder] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:42.416597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e4a8a4-bb47-4c45-b5ca-b2bf2f5d474c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:42.417826 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4a8a4-bb47-4c45-b5ca-b2bf2f5d474c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:42.420584 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4a8a4-bb47-4c45-b5ca-b2bf2f5d474c admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:42.421239 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4a8a4-bb47-4c45-b5ca-b2bf2f5d474c admin admin] PATH_INFO: `/v3/projects/facae1333bf447d792719920c88e54d0` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:42.423372 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e4a8a4-bb47-4c45-b5ca-b2bf2f5d474c admin admin] RBAC: Authorizing `identity:delete_project(project_id=facae1333bf447d792719920c88e54d0)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:42.427821 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e4a8a4-bb47-4c45-b5ca-b2bf2f5d474c admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:42.514601 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-93e4a8a4-bb47-4c45-b5ca-b2bf2f5d474c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'facae1333bf447d792719920c88e54d0', 'request_id': 'req-93e4a8a4-bb47-4c45-b5ca-b2bf2f5d474c'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:42.522666 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-93e4a8a4-bb47-4c45-b5ca-b2bf2f5d474c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'facae1333bf447d792719920c88e54d0', 'request_id': 'req-93e4a8a4-bb47-4c45-b5ca-b2bf2f5d474c'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:42.532878 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-93e4a8a4-bb47-4c45-b5ca-b2bf2f5d474c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'facae1333bf447d792719920c88e54d0', 'request_id': 'req-93e4a8a4-bb47-4c45-b5ca-b2bf2f5d474c'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:42.547509 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-93e4a8a4-bb47-4c45-b5ca-b2bf2f5d474c admin admin] The token cache is being invalidate because project facae1333bf447d792719920c88e54d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:42.548029 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-93e4a8a4-bb47-4c45-b5ca-b2bf2f5d474c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:42.548685 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-93e4a8a4-bb47-4c45-b5ca-b2bf2f5d474c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:42.550491 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 924/1832] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:42 2022] DELETE /identity/v3/projects/facae1333bf447d792719920c88e54d0 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:42.554687 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c04aa-6247-45ef-a968-ede5522e79d2 service neutron] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:42.565481 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c04aa-6247-45ef-a968-ede5522e79d2 service neutron] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:42.654620 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e387df-2419-40c5-b7c5-c54a7d350480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:42.655521 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e387df-2419-40c5-b7c5-c54a7d350480 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:42.655801 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e387df-2419-40c5-b7c5-c54a7d350480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:42.656075 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e387df-2419-40c5-b7c5-c54a7d350480 admin admin] PATH_INFO: `/v3/users/99741385667148aa91e657ebdcd92b66` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:42.657322 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e387df-2419-40c5-b7c5-c54a7d350480 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99741385667148aa91e657ebdcd92b66)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:42.659301 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e387df-2419-40c5-b7c5-c54a7d350480 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:42.725269 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:42.735051 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c6e387df-2419-40c5-b7c5-c54a7d350480 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99741385667148aa91e657ebdcd92b66'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:42.747069 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c6e387df-2419-40c5-b7c5-c54a7d350480 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99741385667148aa91e657ebdcd92b66'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:42.748043 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c6e387df-2419-40c5-b7c5-c54a7d350480 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99741385667148aa91e657ebdcd92b66'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:42.748884 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c6e387df-2419-40c5-b7c5-c54a7d350480 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99741385667148aa91e657ebdcd92b66'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:42.768079 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c6e387df-2419-40c5-b7c5-c54a7d350480 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99741385667148aa91e657ebdcd92b66'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:42.768507 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723d558-abb5-4dfe-af67-becb11e2697b None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:42.768982 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723d558-abb5-4dfe-af67-becb11e2697b None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:42.769419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d723d558-abb5-4dfe-af67-becb11e2697b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:42.798048 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 909/1833] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:16:42 2022] DELETE /identity/v3/users/99741385667148aa91e657ebdcd92b66 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:42.831043 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-d723d558-abb5-4dfe-af67-becb11e2697b None None] MFA Rules not processed for user `2bbf85747f624ab6bfb4b4eec90c3ca8`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:42.832321 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d723d558-abb5-4dfe-af67-becb11e2697b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:42.929697 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:43.013576 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 925/1834] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:42 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:16:43.027648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e387df-2419-40c5-b7c5-c54a7d350480 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:43.029801 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-c6e387df-2419-40c5-b7c5-c54a7d350480 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:43.177445 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e387df-2419-40c5-b7c5-c54a7d350480 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:43.194088 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067bc00-502e-4393-aa11-fb5999795aa0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:43.195434 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067bc00-502e-4393-aa11-fb5999795aa0 service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:43.197974 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067bc00-502e-4393-aa11-fb5999795aa0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:43.198489 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067bc00-502e-4393-aa11-fb5999795aa0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:43.207102 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-1067bc00-502e-4393-aa11-fb5999795aa0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:43.314364 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1067bc00-502e-4393-aa11-fb5999795aa0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:43.315986 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1067bc00-502e-4393-aa11-fb5999795aa0 service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:43.345213 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 910/1835] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:43.575739 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67a684-75a3-49c1-ba64-1ef9bcfe6bf0 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:43.576755 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67a684-75a3-49c1-ba64-1ef9bcfe6bf0 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:43.577250 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67a684-75a3-49c1-ba64-1ef9bcfe6bf0 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:43.578921 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 926/1836] 10.222.0.13 () {54 vars in 921 bytes} [Mon Jun 27 21:16:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:16:43.594789 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09367c68-7b84-4156-b63e-d9236f4ad3ff None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:43.595738 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09367c68-7b84-4156-b63e-d9236f4ad3ff None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:43.596021 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09367c68-7b84-4156-b63e-d9236f4ad3ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:43.619097 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-09367c68-7b84-4156-b63e-d9236f4ad3ff None None] MFA Rules not processed for user `1366958034d540b4ba2feb421a6afabc`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:43.620971 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-09367c68-7b84-4156-b63e-d9236f4ad3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:43.734835 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 911/1837] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Jun 27 21:16:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:43.884349 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c67a684-75a3-49c1-ba64-1ef9bcfe6bf0 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:43.886809 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-7c67a684-75a3-49c1-ba64-1ef9bcfe6bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:44.052819 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c67a684-75a3-49c1-ba64-1ef9bcfe6bf0 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:44.060493 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09367c68-7b84-4156-b63e-d9236f4ad3ff None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:44.062067 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-09367c68-7b84-4156-b63e-d9236f4ad3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:44.064508 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251213bd-1b8a-4432-b943-b70866f7ef5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:44.065806 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251213bd-1b8a-4432-b943-b70866f7ef5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:44.066566 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251213bd-1b8a-4432-b943-b70866f7ef5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:44.066845 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251213bd-1b8a-4432-b943-b70866f7ef5f admin admin] PATH_INFO: `/v3/projects/ca9ba1db2acb4da79b70382068265678` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:44.068807 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251213bd-1b8a-4432-b943-b70866f7ef5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca9ba1db2acb4da79b70382068265678)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:44.079327 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:44.086236 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251213bd-1b8a-4432-b943-b70866f7ef5f admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:44.096441 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09367c68-7b84-4156-b63e-d9236f4ad3ff None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:44.129418 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8439f24-e97e-4177-ab3d-3e3731637937 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:44.130505 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8439f24-e97e-4177-ab3d-3e3731637937 service nova] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:44.130505 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8439f24-e97e-4177-ab3d-3e3731637937 service nova] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:44.130673 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8439f24-e97e-4177-ab3d-3e3731637937 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:44.163225 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8439f24-e97e-4177-ab3d-3e3731637937 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:44.163225 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8439f24-e97e-4177-ab3d-3e3731637937 service nova] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:44.186449 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-251213bd-1b8a-4432-b943-b70866f7ef5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca9ba1db2acb4da79b70382068265678', 'request_id': 'req-251213bd-1b8a-4432-b943-b70866f7ef5f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:44.193138 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 912/1838] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:44.194549 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-251213bd-1b8a-4432-b943-b70866f7ef5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca9ba1db2acb4da79b70382068265678', 'request_id': 'req-251213bd-1b8a-4432-b943-b70866f7ef5f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:44.215160 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-251213bd-1b8a-4432-b943-b70866f7ef5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca9ba1db2acb4da79b70382068265678', 'request_id': 'req-251213bd-1b8a-4432-b943-b70866f7ef5f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:44.241730 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-251213bd-1b8a-4432-b943-b70866f7ef5f admin admin] The token cache is being invalidate because project ca9ba1db2acb4da79b70382068265678 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:44.242840 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-251213bd-1b8a-4432-b943-b70866f7ef5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:44.244643 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-251213bd-1b8a-4432-b943-b70866f7ef5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:44.246847 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 927/1839] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:43 2022] DELETE /identity/v3/projects/ca9ba1db2acb4da79b70382068265678 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:44.251824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde35fb-0e21-4de3-aba6-c6460ce34946 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:44.252119 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde35fb-0e21-4de3-aba6-c6460ce34946 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:44.252394 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde35fb-0e21-4de3-aba6-c6460ce34946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:44.287984 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-0dde35fb-0e21-4de3-aba6-c6460ce34946 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:44.289260 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-0dde35fb-0e21-4de3-aba6-c6460ce34946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:44.320779 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251213bd-1b8a-4432-b943-b70866f7ef5f admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:44.322979 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-251213bd-1b8a-4432-b943-b70866f7ef5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:44.451126 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 913/1840] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:44.459580 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b2c16-15c1-4375-973d-ab2af62edc47 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:44.459868 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b2c16-15c1-4375-973d-ab2af62edc47 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:44.460155 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b2c16-15c1-4375-973d-ab2af62edc47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:44.471579 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251213bd-1b8a-4432-b943-b70866f7ef5f admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:44.481218 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa6213-ab87-4893-a7f9-d8b77fe20786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:44.482467 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa6213-ab87-4893-a7f9-d8b77fe20786 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:44.482906 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa6213-ab87-4893-a7f9-d8b77fe20786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:44.483386 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa6213-ab87-4893-a7f9-d8b77fe20786 admin admin] PATH_INFO: `/v3/users/3a9d25cbfa5e4093b29907453a632703` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:44.484961 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aa6213-ab87-4893-a7f9-d8b77fe20786 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a9d25cbfa5e4093b29907453a632703)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:44.485339 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-845b2c16-15c1-4375-973d-ab2af62edc47 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:44.487170 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-845b2c16-15c1-4375-973d-ab2af62edc47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:44.487903 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aa6213-ab87-4893-a7f9-d8b77fe20786 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:44.497628 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 914/1841] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:16:44.501566 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b2c16-15c1-4375-973d-ab2af62edc47 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:44.514450 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:44.515215 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b2c16-15c1-4375-973d-ab2af62edc47 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:44.524027 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad7754-e23d-4d3f-bb14-e45a77829142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:44.525007 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad7754-e23d-4d3f-bb14-e45a77829142 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:44.525214 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad7754-e23d-4d3f-bb14-e45a77829142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:44.525492 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad7754-e23d-4d3f-bb14-e45a77829142 admin admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:44.527197 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad7754-e23d-4d3f-bb14-e45a77829142 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:16:44.527455 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad7754-e23d-4d3f-bb14-e45a77829142 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:44.529395 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad7754-e23d-4d3f-bb14-e45a77829142 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:44.544496 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 915/1842] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:16:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:16:44.549347 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad7754-e23d-4d3f-bb14-e45a77829142 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:44.564795 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad7754-e23d-4d3f-bb14-e45a77829142 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:44.571513 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ab1c0-140e-4912-8661-aa073a0d6d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:44.572549 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ab1c0-140e-4912-8661-aa073a0d6d8f admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:44.572995 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ab1c0-140e-4912-8661-aa073a0d6d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:44.573512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ab1c0-140e-4912-8661-aa073a0d6d8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:44.575528 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ab1c0-140e-4912-8661-aa073a0d6d8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:44.577283 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ab1c0-140e-4912-8661-aa073a0d6d8f admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:44.611508 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-62aa6213-ab87-4893-a7f9-d8b77fe20786 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a9d25cbfa5e4093b29907453a632703'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:44.620289 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-62aa6213-ab87-4893-a7f9-d8b77fe20786 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a9d25cbfa5e4093b29907453a632703'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:44.620736 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 916/1843] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:44 2022] POST /identity/v3/projects => generated 389 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:44.622909 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-62aa6213-ab87-4893-a7f9-d8b77fe20786 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a9d25cbfa5e4093b29907453a632703'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:44.624082 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-62aa6213-ab87-4893-a7f9-d8b77fe20786 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a9d25cbfa5e4093b29907453a632703'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:44.625966 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ab1c0-140e-4912-8661-aa073a0d6d8f admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:44.628030 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-e17ab1c0-140e-4912-8661-aa073a0d6d8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:44.636159 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-62aa6213-ab87-4893-a7f9-d8b77fe20786 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a9d25cbfa5e4093b29907453a632703'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:44.676056 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 928/1844] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:16:44 2022] DELETE /identity/v3/users/3a9d25cbfa5e4093b29907453a632703 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:44.680936 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735d189-bd34-4507-81de-c161b0391738 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:44.681492 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735d189-bd34-4507-81de-c161b0391738 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:44.681942 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735d189-bd34-4507-81de-c161b0391738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:44.701262 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-7735d189-bd34-4507-81de-c161b0391738 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:44.702490 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-7735d189-bd34-4507-81de-c161b0391738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:44.733336 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ab1c0-140e-4912-8661-aa073a0d6d8f admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:44.820138 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 929/1845] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:16:44.831469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7735d189-bd34-4507-81de-c161b0391738 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:44.832421 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-7735d189-bd34-4507-81de-c161b0391738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:44.837285 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:44.846984 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e1a783-6066-49f7-a287-5338e8275360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:44.847822 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1a783-6066-49f7-a287-5338e8275360 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:44.848441 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1a783-6066-49f7-a287-5338e8275360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:44.849193 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1a783-6066-49f7-a287-5338e8275360 admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:44.850361 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e1a783-6066-49f7-a287-5338e8275360 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:44.852480 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e1a783-6066-49f7-a287-5338e8275360 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:44.880961 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 917/1846] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:44 2022] POST /identity/v3/users => generated 295 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:44.885421 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e1a783-6066-49f7-a287-5338e8275360 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:44.900030 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7735d189-bd34-4507-81de-c161b0391738 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:44.900812 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e1a783-6066-49f7-a287-5338e8275360 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:44.904109 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e5c78c-aa98-48ae-876d-d14a22141464 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:44.904786 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e5c78c-aa98-48ae-876d-d14a22141464 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:44.905137 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e5c78c-aa98-48ae-876d-d14a22141464 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:44.905448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e5c78c-aa98-48ae-876d-d14a22141464 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:44.906452 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975fdd71-399a-489d-bc61-b44625912655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:44.907139 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975fdd71-399a-489d-bc61-b44625912655 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:44.907484 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975fdd71-399a-489d-bc61-b44625912655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:44.907771 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975fdd71-399a-489d-bc61-b44625912655 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:44.908516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975fdd71-399a-489d-bc61-b44625912655 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:44.910218 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975fdd71-399a-489d-bc61-b44625912655 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:44.916865 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e5c78c-aa98-48ae-876d-d14a22141464 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:44.918323 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e5c78c-aa98-48ae-876d-d14a22141464 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:44.923097 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 918/1847] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:44.932983 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 930/1848] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:16:44.934430 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975fdd71-399a-489d-bc61-b44625912655 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:44.966963 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975fdd71-399a-489d-bc61-b44625912655 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:44.974926 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7acff9e-4707-4402-b1dd-be6033e6318d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:44.975638 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7acff9e-4707-4402-b1dd-be6033e6318d admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:44.975922 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7acff9e-4707-4402-b1dd-be6033e6318d admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:44.976196 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7acff9e-4707-4402-b1dd-be6033e6318d admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:44.977438 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7acff9e-4707-4402-b1dd-be6033e6318d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:44.979545 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7acff9e-4707-4402-b1dd-be6033e6318d admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:44.990372 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e5c78c-aa98-48ae-876d-d14a22141464 service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:44.992483 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d8e5c78c-aa98-48ae-876d-d14a22141464 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:44.997069 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 919/1849] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:45.001973 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7acff9e-4707-4402-b1dd-be6033e6318d admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:45.015338 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7acff9e-4707-4402-b1dd-be6033e6318d admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:45.032699 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a304b7-90c6-48ad-bff4-b9bf32cac197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:45.034065 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a304b7-90c6-48ad-bff4-b9bf32cac197 admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:45.034588 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a304b7-90c6-48ad-bff4-b9bf32cac197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:45.035077 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a304b7-90c6-48ad-bff4-b9bf32cac197 admin admin] PATH_INFO: `/v3/projects/a8b4f817b53c45b680475a6fe07eacec/users/d2b04a09732a440b868538c774430a78/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:45.047178 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:45.061858 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a304b7-90c6-48ad-bff4-b9bf32cac197 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8b4f817b53c45b680475a6fe07eacec, user_id=d2b04a09732a440b868538c774430a78, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:45.067911 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a304b7-90c6-48ad-bff4-b9bf32cac197 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:45.099044 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 920/1850] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:45 2022] PUT /identity/v3/projects/a8b4f817b53c45b680475a6fe07eacec/users/d2b04a09732a440b868538c774430a78/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:45.100244 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f342788-f1d0-4680-9643-034adfde3b96 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:45.101359 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f342788-f1d0-4680-9643-034adfde3b96 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:45.104311 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f342788-f1d0-4680-9643-034adfde3b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:45.138548 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-0f342788-f1d0-4680-9643-034adfde3b96 None None] MFA Rules not processed for user `d2b04a09732a440b868538c774430a78`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:45.140128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-0f342788-f1d0-4680-9643-034adfde3b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:45.216966 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e5c78c-aa98-48ae-876d-d14a22141464 service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:45.263924 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 921/1851] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:45.267106 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f342788-f1d0-4680-9643-034adfde3b96 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:45.287202 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa57c1-3d74-49b2-a4e8-e6d5ce120469 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:45.287469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa57c1-3d74-49b2-a4e8-e6d5ce120469 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:45.287657 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa57c1-3d74-49b2-a4e8-e6d5ce120469 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:45.288081 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa57c1-3d74-49b2-a4e8-e6d5ce120469 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:45.291220 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-f9fa57c1-3d74-49b2-a4e8-e6d5ce120469 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:45.373190 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fa57c1-3d74-49b2-a4e8-e6d5ce120469 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:45.375279 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fa57c1-3d74-49b2-a4e8-e6d5ce120469 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:45.390122 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 931/1852] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:44 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:16:45.425790 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f342788-f1d0-4680-9643-034adfde3b96 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:45.433176 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180662c-8623-4660-a0f0-e9bd9065f05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:45.434474 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180662c-8623-4660-a0f0-e9bd9065f05a admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:45.434832 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180662c-8623-4660-a0f0-e9bd9065f05a admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:45.435329 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180662c-8623-4660-a0f0-e9bd9065f05a admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:45.436939 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3180662c-8623-4660-a0f0-e9bd9065f05a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:45.441312 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3180662c-8623-4660-a0f0-e9bd9065f05a admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:45.484765 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 922/1853] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:45 2022] POST /identity/v3/projects => generated 389 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:45.484970 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa57c1-3d74-49b2-a4e8-e6d5ce120469 service cinder] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:45.501692 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa57c1-3d74-49b2-a4e8-e6d5ce120469 service cinder] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:45.654781 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa5079c-2b0d-43b3-acd5-d60536d8490a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:45.655462 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5079c-2b0d-43b3-acd5-d60536d8490a admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:45.655660 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5079c-2b0d-43b3-acd5-d60536d8490a admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:45.655940 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5079c-2b0d-43b3-acd5-d60536d8490a admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:45.656651 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa5079c-2b0d-43b3-acd5-d60536d8490a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:45.658732 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa5079c-2b0d-43b3-acd5-d60536d8490a admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:45.709470 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 932/1854] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:45 2022] POST /identity/v3/users => generated 301 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:16:45.711068 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180662c-8623-4660-a0f0-e9bd9065f05a admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:45.729082 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180662c-8623-4660-a0f0-e9bd9065f05a admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:45.736889 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ea1ab-6f1d-4659-86c1-7775bacac080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:45.738652 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ea1ab-6f1d-4659-86c1-7775bacac080 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:45.738953 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ea1ab-6f1d-4659-86c1-7775bacac080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:45.739229 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ea1ab-6f1d-4659-86c1-7775bacac080 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:45.741695 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488ea1ab-6f1d-4659-86c1-7775bacac080 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:45.744005 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488ea1ab-6f1d-4659-86c1-7775bacac080 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:45.765070 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 923/1855] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:45 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:45.765886 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa5079c-2b0d-43b3-acd5-d60536d8490a admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:45.778382 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa5079c-2b0d-43b3-acd5-d60536d8490a admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:45.793289 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ddb612-46ef-4be6-8cff-a4f1f56e7719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:45.794702 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ddb612-46ef-4be6-8cff-a4f1f56e7719 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:45.795674 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ddb612-46ef-4be6-8cff-a4f1f56e7719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:45.796120 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ddb612-46ef-4be6-8cff-a4f1f56e7719 admin admin] PATH_INFO: `/v3/projects/cbc73a3a9d0445b785c05e9e00d1c431/users/e4fb0403fc454fa497686f7a9fa9998a/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:45.833644 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ddb612-46ef-4be6-8cff-a4f1f56e7719 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbc73a3a9d0445b785c05e9e00d1c431, user_id=e4fb0403fc454fa497686f7a9fa9998a, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:45.837105 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ddb612-46ef-4be6-8cff-a4f1f56e7719 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:45.858922 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 933/1856] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:45 2022] PUT /identity/v3/projects/cbc73a3a9d0445b785c05e9e00d1c431/users/e4fb0403fc454fa497686f7a9fa9998a/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:45.863049 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ea1ab-6f1d-4659-86c1-7775bacac080 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:45.882240 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:45.899656 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ea1ab-6f1d-4659-86c1-7775bacac080 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:46.063181 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db09380-aeb2-46ee-925e-774815cff033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:46.063181 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db09380-aeb2-46ee-925e-774815cff033 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:46.063630 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db09380-aeb2-46ee-925e-774815cff033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:46.063849 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db09380-aeb2-46ee-925e-774815cff033 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:46.064840 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db09380-aeb2-46ee-925e-774815cff033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:46.066516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db09380-aeb2-46ee-925e-774815cff033 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:46.080251 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 924/1857] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:45 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:46.085285 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ddb612-46ef-4be6-8cff-a4f1f56e7719 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:46.093063 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:46.097875 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ddb612-46ef-4be6-8cff-a4f1f56e7719 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:46.104157 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06b82f0-3c68-414c-8ffe-05f999478b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:46.105399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b82f0-3c68-414c-8ffe-05f999478b31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:46.105876 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b82f0-3c68-414c-8ffe-05f999478b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:46.106279 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b82f0-3c68-414c-8ffe-05f999478b31 admin admin] PATH_INFO: `/v3/projects/cbc73a3a9d0445b785c05e9e00d1c431/users/e4fb0403fc454fa497686f7a9fa9998a/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:46.108420 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06b82f0-3c68-414c-8ffe-05f999478b31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbc73a3a9d0445b785c05e9e00d1c431, user_id=e4fb0403fc454fa497686f7a9fa9998a, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:46.112054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06b82f0-3c68-414c-8ffe-05f999478b31 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:46.124360 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 934/1858] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:46 2022] PUT /identity/v3/projects/cbc73a3a9d0445b785c05e9e00d1c431/users/e4fb0403fc454fa497686f7a9fa9998a/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:46.135307 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b7d20-1c0c-4deb-bc57-80f1cba8d245 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:46.135603 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b7d20-1c0c-4deb-bc57-80f1cba8d245 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:46.135895 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b7d20-1c0c-4deb-bc57-80f1cba8d245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:46.170578 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-4f7b7d20-1c0c-4deb-bc57-80f1cba8d245 None None] MFA Rules not processed for user `e4fb0403fc454fa497686f7a9fa9998a`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:46.172212 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-4f7b7d20-1c0c-4deb-bc57-80f1cba8d245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:46.173550 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06b82f0-3c68-414c-8ffe-05f999478b31 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:46.175991 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-e06b82f0-3c68-414c-8ffe-05f999478b31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:46.261645 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06b82f0-3c68-414c-8ffe-05f999478b31 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:46.272658 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d7b2a-393c-4146-b2ab-fb17fe593f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:46.273185 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 925/1859] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:46 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:46.273814 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d7b2a-393c-4146-b2ab-fb17fe593f2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:46.274172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d7b2a-393c-4146-b2ab-fb17fe593f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:46.274607 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d7b2a-393c-4146-b2ab-fb17fe593f2a admin admin] PATH_INFO: `/v3/projects/9cf6a289a0834a4a8b02325cd9e00ab6` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:46.276197 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07d7b2a-393c-4146-b2ab-fb17fe593f2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cf6a289a0834a4a8b02325cd9e00ab6)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:46.278723 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07d7b2a-393c-4146-b2ab-fb17fe593f2a admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:46.288657 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b7d20-1c0c-4deb-bc57-80f1cba8d245 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:46.346802 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-d07d7b2a-393c-4146-b2ab-fb17fe593f2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cf6a289a0834a4a8b02325cd9e00ab6', 'request_id': 'req-d07d7b2a-393c-4146-b2ab-fb17fe593f2a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:46.352385 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b7d20-1c0c-4deb-bc57-80f1cba8d245 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:46.353894 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-d07d7b2a-393c-4146-b2ab-fb17fe593f2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cf6a289a0834a4a8b02325cd9e00ab6', 'request_id': 'req-d07d7b2a-393c-4146-b2ab-fb17fe593f2a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:46.363453 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-d07d7b2a-393c-4146-b2ab-fb17fe593f2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cf6a289a0834a4a8b02325cd9e00ab6', 'request_id': 'req-d07d7b2a-393c-4146-b2ab-fb17fe593f2a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:46.382533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-d07d7b2a-393c-4146-b2ab-fb17fe593f2a admin admin] The token cache is being invalidate because project 9cf6a289a0834a4a8b02325cd9e00ab6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:46.382826 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-d07d7b2a-393c-4146-b2ab-fb17fe593f2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:46.383515 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-d07d7b2a-393c-4146-b2ab-fb17fe593f2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:46.386427 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 935/1860] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:46 2022] DELETE /identity/v3/projects/9cf6a289a0834a4a8b02325cd9e00ab6 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:46.390693 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d7b2a-393c-4146-b2ab-fb17fe593f2a admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:46.392194 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d07d7b2a-393c-4146-b2ab-fb17fe593f2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:46.419289 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:46.434704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220e73b1-c3d0-484a-9ce9-c1218155eea2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:46.435432 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e73b1-c3d0-484a-9ce9-c1218155eea2 service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:46.435788 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e73b1-c3d0-484a-9ce9-c1218155eea2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:46.435981 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e73b1-c3d0-484a-9ce9-c1218155eea2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:46.438253 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-220e73b1-c3d0-484a-9ce9-c1218155eea2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:46.481679 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d7b2a-393c-4146-b2ab-fb17fe593f2a admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:46.486891 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b129dc4-57eb-4314-9920-fe26082f2937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:46.487988 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b129dc4-57eb-4314-9920-fe26082f2937 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:46.488272 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b129dc4-57eb-4314-9920-fe26082f2937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:46.488545 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b129dc4-57eb-4314-9920-fe26082f2937 admin admin] PATH_INFO: `/v3/users/fda3d835da0d429bbf44a17fbd502f4a` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:46.490669 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b129dc4-57eb-4314-9920-fe26082f2937 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fda3d835da0d429bbf44a17fbd502f4a)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:46.492344 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b129dc4-57eb-4314-9920-fe26082f2937 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:46.521901 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:46.533474 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220e73b1-c3d0-484a-9ce9-c1218155eea2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:46.535040 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220e73b1-c3d0-484a-9ce9-c1218155eea2 service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:46.548491 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 926/1861] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:46 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:46.562166 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6b129dc4-57eb-4314-9920-fe26082f2937 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fda3d835da0d429bbf44a17fbd502f4a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:46.575085 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6b129dc4-57eb-4314-9920-fe26082f2937 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fda3d835da0d429bbf44a17fbd502f4a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:46.577933 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6b129dc4-57eb-4314-9920-fe26082f2937 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fda3d835da0d429bbf44a17fbd502f4a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:46.578579 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6b129dc4-57eb-4314-9920-fe26082f2937 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fda3d835da0d429bbf44a17fbd502f4a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:46.591762 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6b129dc4-57eb-4314-9920-fe26082f2937 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fda3d835da0d429bbf44a17fbd502f4a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:46.616375 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 936/1862] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:16:46 2022] DELETE /identity/v3/users/fda3d835da0d429bbf44a17fbd502f4a => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:16:46.698579 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220e73b1-c3d0-484a-9ce9-c1218155eea2 service cinder] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:46.700008 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-220e73b1-c3d0-484a-9ce9-c1218155eea2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:46.717163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220e73b1-c3d0-484a-9ce9-c1218155eea2 service cinder] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:46.806624 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce9a75-ef6d-4a21-98e3-0975558a3115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:46.807815 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce9a75-ef6d-4a21-98e3-0975558a3115 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:46.808101 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce9a75-ef6d-4a21-98e3-0975558a3115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:46.808439 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce9a75-ef6d-4a21-98e3-0975558a3115 admin admin] PATH_INFO: `/v3/users/d2b04a09732a440b868538c774430a78` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:46.809841 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce9a75-ef6d-4a21-98e3-0975558a3115 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2b04a09732a440b868538c774430a78)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:46.812959 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce9a75-ef6d-4a21-98e3-0975558a3115 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:46.919464 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-5bce9a75-ef6d-4a21-98e3-0975558a3115 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2b04a09732a440b868538c774430a78'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:46.952847 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-5bce9a75-ef6d-4a21-98e3-0975558a3115 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2b04a09732a440b868538c774430a78'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:46.955727 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-5bce9a75-ef6d-4a21-98e3-0975558a3115 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2b04a09732a440b868538c774430a78'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:46.958589 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-5bce9a75-ef6d-4a21-98e3-0975558a3115 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2b04a09732a440b868538c774430a78'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:46.983172 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-5bce9a75-ef6d-4a21-98e3-0975558a3115 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2b04a09732a440b868538c774430a78'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:47.010746 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 927/1863] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:16:46 2022] DELETE /identity/v3/users/d2b04a09732a440b868538c774430a78 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:47.019717 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99115be1-ac6c-410d-87f2-a78d6118c514 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:47.020284 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99115be1-ac6c-410d-87f2-a78d6118c514 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:47.020391 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99115be1-ac6c-410d-87f2-a78d6118c514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:47.059175 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-99115be1-ac6c-410d-87f2-a78d6118c514 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:47.060448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-99115be1-ac6c-410d-87f2-a78d6118c514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:47.067469 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce9a75-ef6d-4a21-98e3-0975558a3115 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:47.069648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-5bce9a75-ef6d-4a21-98e3-0975558a3115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:47.089443 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:47.190347 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce9a75-ef6d-4a21-98e3-0975558a3115 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:47.196865 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c887a70-cc58-4061-8c76-3c3451a15e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:47.197791 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c887a70-cc58-4061-8c76-3c3451a15e7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:47.198804 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c887a70-cc58-4061-8c76-3c3451a15e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:47.199229 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c887a70-cc58-4061-8c76-3c3451a15e7d admin admin] PATH_INFO: `/v3/projects/3050c89ed098482c81c0f6b3f0f8b451` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:47.201480 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c887a70-cc58-4061-8c76-3c3451a15e7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3050c89ed098482c81c0f6b3f0f8b451)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:47.203570 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c887a70-cc58-4061-8c76-3c3451a15e7d admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:47.251514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 937/1864] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:47.260099 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99115be1-ac6c-410d-87f2-a78d6118c514 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:47.261635 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-99115be1-ac6c-410d-87f2-a78d6118c514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:47.305696 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-8c887a70-cc58-4061-8c76-3c3451a15e7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3050c89ed098482c81c0f6b3f0f8b451', 'request_id': 'req-8c887a70-cc58-4061-8c76-3c3451a15e7d'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:47.320577 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-8c887a70-cc58-4061-8c76-3c3451a15e7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3050c89ed098482c81c0f6b3f0f8b451', 'request_id': 'req-8c887a70-cc58-4061-8c76-3c3451a15e7d'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:47.344374 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-8c887a70-cc58-4061-8c76-3c3451a15e7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3050c89ed098482c81c0f6b3f0f8b451', 'request_id': 'req-8c887a70-cc58-4061-8c76-3c3451a15e7d'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:47.349190 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99115be1-ac6c-410d-87f2-a78d6118c514 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:47.372113 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-8c887a70-cc58-4061-8c76-3c3451a15e7d admin admin] The token cache is being invalidate because project 3050c89ed098482c81c0f6b3f0f8b451 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:47.372113 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-8c887a70-cc58-4061-8c76-3c3451a15e7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:47.373473 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-8c887a70-cc58-4061-8c76-3c3451a15e7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:47.378299 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 928/1865] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:47 2022] DELETE /identity/v3/projects/3050c89ed098482c81c0f6b3f0f8b451 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:47.384880 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de3bbd-706e-4a13-a48c-296690a7652b None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:47.386541 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de3bbd-706e-4a13-a48c-296690a7652b None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:47.387882 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de3bbd-706e-4a13-a48c-296690a7652b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:47.418967 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-20de3bbd-706e-4a13-a48c-296690a7652b None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:47.420254 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-20de3bbd-706e-4a13-a48c-296690a7652b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:47.448135 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3fbe30-0dc5-4b5c-a5ae-71e2a1c026f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:47.448816 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fbe30-0dc5-4b5c-a5ae-71e2a1c026f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:47.449208 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fbe30-0dc5-4b5c-a5ae-71e2a1c026f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:47.450140 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3fbe30-0dc5-4b5c-a5ae-71e2a1c026f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:47.452111 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-4a3fbe30-0dc5-4b5c-a5ae-71e2a1c026f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:47.473055 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:47.520552 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 929/1866] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:16:47.526186 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0560ce-a8c2-43b6-8804-16d66de89e3c None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:47.527458 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0560ce-a8c2-43b6-8804-16d66de89e3c None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:47.528025 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0560ce-a8c2-43b6-8804-16d66de89e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:47.565225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3fbe30-0dc5-4b5c-a5ae-71e2a1c026f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:47.565450 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-4f0560ce-a8c2-43b6-8804-16d66de89e3c None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:47.566907 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3fbe30-0dc5-4b5c-a5ae-71e2a1c026f1 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:47.567069 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-4f0560ce-a8c2-43b6-8804-16d66de89e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:47.578324 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 930/1867] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:47.579988 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 938/1868] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:16:47.583667 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0560ce-a8c2-43b6-8804-16d66de89e3c None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:47.594473 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0560ce-a8c2-43b6-8804-16d66de89e3c None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:47.599716 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a7a498-92d9-49b7-bb3d-68e414c2d8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:47.600555 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7a498-92d9-49b7-bb3d-68e414c2d8b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:47.600838 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7a498-92d9-49b7-bb3d-68e414c2d8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:47.601168 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7a498-92d9-49b7-bb3d-68e414c2d8b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:47.602063 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a7a498-92d9-49b7-bb3d-68e414c2d8b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:16:47.602367 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a7a498-92d9-49b7-bb3d-68e414c2d8b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:47.605694 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a7a498-92d9-49b7-bb3d-68e414c2d8b7 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:47.624988 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 931/1869] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:16:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:16:47.626636 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3fbe30-0dc5-4b5c-a5ae-71e2a1c026f1 service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:47.640070 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3fbe30-0dc5-4b5c-a5ae-71e2a1c026f1 service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:47.646783 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b02c9-6f5b-4f56-abfc-06579b9e7773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:47.648280 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b02c9-6f5b-4f56-abfc-06579b9e7773 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:47.648613 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b02c9-6f5b-4f56-abfc-06579b9e7773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:47.648899 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b02c9-6f5b-4f56-abfc-06579b9e7773 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:47.650613 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62b02c9-6f5b-4f56-abfc-06579b9e7773 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:47.653713 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62b02c9-6f5b-4f56-abfc-06579b9e7773 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:47.690778 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 939/1870] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:47 2022] POST /identity/v3/projects => generated 377 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:47.695156 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a7a498-92d9-49b7-bb3d-68e414c2d8b7 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:47.710745 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a7a498-92d9-49b7-bb3d-68e414c2d8b7 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:47.749951 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:47.862078 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce299c-e9e4-45fd-8089-1537807bc36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:47.862903 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce299c-e9e4-45fd-8089-1537807bc36f admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:47.863525 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce299c-e9e4-45fd-8089-1537807bc36f admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:47.863906 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce299c-e9e4-45fd-8089-1537807bc36f admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:47.864768 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ce299c-e9e4-45fd-8089-1537807bc36f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:47.867526 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ce299c-e9e4-45fd-8089-1537807bc36f admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:47.891935 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 932/1871] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:47 2022] POST /identity/v3/users => generated 289 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:47.895910 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b02c9-6f5b-4f56-abfc-06579b9e7773 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:47.906838 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b02c9-6f5b-4f56-abfc-06579b9e7773 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:47.913034 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf7eb56-2525-40c8-b94a-5b88c29ab55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:47.913781 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7eb56-2525-40c8-b94a-5b88c29ab55f admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:47.914154 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7eb56-2525-40c8-b94a-5b88c29ab55f admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:47.915334 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7eb56-2525-40c8-b94a-5b88c29ab55f admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:47.916408 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf7eb56-2525-40c8-b94a-5b88c29ab55f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:47.918322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf7eb56-2525-40c8-b94a-5b88c29ab55f admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:47.951986 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 940/1872] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:47.957006 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce299c-e9e4-45fd-8089-1537807bc36f admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:47.971713 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce299c-e9e4-45fd-8089-1537807bc36f admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:47.978821 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba443dbe-2e9c-4816-a2e7-9e7deaeba41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:47.981403 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba443dbe-2e9c-4816-a2e7-9e7deaeba41b admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:47.982176 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba443dbe-2e9c-4816-a2e7-9e7deaeba41b admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:47.982567 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba443dbe-2e9c-4816-a2e7-9e7deaeba41b admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:47.984241 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba443dbe-2e9c-4816-a2e7-9e7deaeba41b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:47.986120 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba443dbe-2e9c-4816-a2e7-9e7deaeba41b admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:48.003585 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 933/1873] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:48.008302 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf7eb56-2525-40c8-b94a-5b88c29ab55f admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:48.023466 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba443dbe-2e9c-4816-a2e7-9e7deaeba41b admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:48.027749 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf7eb56-2525-40c8-b94a-5b88c29ab55f admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:48.030455 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-ba443dbe-2e9c-4816-a2e7-9e7deaeba41b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:48.041557 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f70063-60e6-44dd-a84d-c0c580afcc95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:48.042919 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f70063-60e6-44dd-a84d-c0c580afcc95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:48.045216 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f70063-60e6-44dd-a84d-c0c580afcc95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:48.045611 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f70063-60e6-44dd-a84d-c0c580afcc95 admin admin] PATH_INFO: `/v3/projects/201d4043c8d9474ebc87b0fe763cc1f9/users/4482c4806e724f899c890e10d0223f0d/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:48.059025 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba443dbe-2e9c-4816-a2e7-9e7deaeba41b admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:48.070425 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d65830b-0bb8-45e2-bcc1-48d9159e28a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:48.072295 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65830b-0bb8-45e2-bcc1-48d9159e28a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:48.072777 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65830b-0bb8-45e2-bcc1-48d9159e28a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:48.073272 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65830b-0bb8-45e2-bcc1-48d9159e28a3 admin admin] PATH_INFO: `/v3/projects/a8b4f817b53c45b680475a6fe07eacec` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:48.074988 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d65830b-0bb8-45e2-bcc1-48d9159e28a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8b4f817b53c45b680475a6fe07eacec)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:48.075945 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f70063-60e6-44dd-a84d-c0c580afcc95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=201d4043c8d9474ebc87b0fe763cc1f9, user_id=4482c4806e724f899c890e10d0223f0d, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:48.077683 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d65830b-0bb8-45e2-bcc1-48d9159e28a3 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:48.079277 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f70063-60e6-44dd-a84d-c0c580afcc95 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:48.099716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 941/1874] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:48 2022] PUT /identity/v3/projects/201d4043c8d9474ebc87b0fe763cc1f9/users/4482c4806e724f899c890e10d0223f0d/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:48.114109 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb22f6-7944-4a1e-847a-772edd5eb3d8 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:48.114589 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb22f6-7944-4a1e-847a-772edd5eb3d8 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:48.114958 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb22f6-7944-4a1e-847a-772edd5eb3d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:48.143475 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-4d65830b-0bb8-45e2-bcc1-48d9159e28a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8b4f817b53c45b680475a6fe07eacec', 'request_id': 'req-4d65830b-0bb8-45e2-bcc1-48d9159e28a3'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:48.151938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-d9cb22f6-7944-4a1e-847a-772edd5eb3d8 None None] MFA Rules not processed for user `4482c4806e724f899c890e10d0223f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:48.153491 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d9cb22f6-7944-4a1e-847a-772edd5eb3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:48.153654 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-4d65830b-0bb8-45e2-bcc1-48d9159e28a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8b4f817b53c45b680475a6fe07eacec', 'request_id': 'req-4d65830b-0bb8-45e2-bcc1-48d9159e28a3'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:48.163512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-4d65830b-0bb8-45e2-bcc1-48d9159e28a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8b4f817b53c45b680475a6fe07eacec', 'request_id': 'req-4d65830b-0bb8-45e2-bcc1-48d9159e28a3'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:48.185491 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-4d65830b-0bb8-45e2-bcc1-48d9159e28a3 admin admin] The token cache is being invalidate because project a8b4f817b53c45b680475a6fe07eacec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:48.185491 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-4d65830b-0bb8-45e2-bcc1-48d9159e28a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:48.190184 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-4d65830b-0bb8-45e2-bcc1-48d9159e28a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:48.190184 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 934/1875] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:48 2022] DELETE /identity/v3/projects/a8b4f817b53c45b680475a6fe07eacec => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:48.192505 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d65830b-0bb8-45e2-bcc1-48d9159e28a3 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:48.194864 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-4d65830b-0bb8-45e2-bcc1-48d9159e28a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:48.236960 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 942/1876] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:16:48.248735 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cb22f6-7944-4a1e-847a-772edd5eb3d8 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:48.250651 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d9cb22f6-7944-4a1e-847a-772edd5eb3d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:48.280462 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d65830b-0bb8-45e2-bcc1-48d9159e28a3 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:48.285697 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12fc22b-b861-4c39-a0a2-0a2784852aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:48.286700 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12fc22b-b861-4c39-a0a2-0a2784852aea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:48.287057 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12fc22b-b861-4c39-a0a2-0a2784852aea admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:48.287490 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12fc22b-b861-4c39-a0a2-0a2784852aea admin admin] PATH_INFO: `/v3/users/e4fb0403fc454fa497686f7a9fa9998a` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:48.289130 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12fc22b-b861-4c39-a0a2-0a2784852aea admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4fb0403fc454fa497686f7a9fa9998a)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:48.291504 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12fc22b-b861-4c39-a0a2-0a2784852aea admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:48.310457 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:48.348731 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cb22f6-7944-4a1e-847a-772edd5eb3d8 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:48.350991 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=85,keystone:INSERT=4,keystone:DELETE=11 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:48.359984 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8909ac8-0af9-4cf2-a8e5-487655a58f32 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26cf962d151b4da7a82c33ef23b342e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:48.361039 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8909ac8-0af9-4cf2-a8e5-487655a58f32 service glance] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:48.361937 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8909ac8-0af9-4cf2-a8e5-487655a58f32 service glance] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:48.362282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8909ac8-0af9-4cf2-a8e5-487655a58f32 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:48.378945 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8909ac8-0af9-4cf2-a8e5-487655a58f32 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:48.381152 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8909ac8-0af9-4cf2-a8e5-487655a58f32 service glance] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:48.382244 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-b12fc22b-b861-4c39-a0a2-0a2784852aea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4fb0403fc454fa497686f7a9fa9998a'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:48.391707 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-b12fc22b-b861-4c39-a0a2-0a2784852aea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4fb0403fc454fa497686f7a9fa9998a'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:48.392545 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-b12fc22b-b861-4c39-a0a2-0a2784852aea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4fb0403fc454fa497686f7a9fa9998a'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:48.393272 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-b12fc22b-b861-4c39-a0a2-0a2784852aea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4fb0403fc454fa497686f7a9fa9998a'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:48.402151 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 943/1877] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:48.405247 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-b12fc22b-b861-4c39-a0a2-0a2784852aea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4fb0403fc454fa497686f7a9fa9998a'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:48.435072 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 935/1878] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:16:48 2022] DELETE /identity/v3/users/e4fb0403fc454fa497686f7a9fa9998a => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:48.949514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8909ac8-0af9-4cf2-a8e5-487655a58f32 service glance] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:48.964247 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8909ac8-0af9-4cf2-a8e5-487655a58f32 service glance] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:49.057536 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c5cbd-0d74-4c95-a75d-590f1d55c337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:49.059149 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c5cbd-0d74-4c95-a75d-590f1d55c337 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:49.059149 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c5cbd-0d74-4c95-a75d-590f1d55c337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:49.059525 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c5cbd-0d74-4c95-a75d-590f1d55c337 admin admin] PATH_INFO: `/v3/projects/cbc73a3a9d0445b785c05e9e00d1c431` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:49.061165 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599c5cbd-0d74-4c95-a75d-590f1d55c337 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbc73a3a9d0445b785c05e9e00d1c431)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:49.063647 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599c5cbd-0d74-4c95-a75d-590f1d55c337 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:49.152860 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-599c5cbd-0d74-4c95-a75d-590f1d55c337 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbc73a3a9d0445b785c05e9e00d1c431', 'request_id': 'req-599c5cbd-0d74-4c95-a75d-590f1d55c337'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:49.160465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-599c5cbd-0d74-4c95-a75d-590f1d55c337 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbc73a3a9d0445b785c05e9e00d1c431', 'request_id': 'req-599c5cbd-0d74-4c95-a75d-590f1d55c337'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:49.177135 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-599c5cbd-0d74-4c95-a75d-590f1d55c337 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbc73a3a9d0445b785c05e9e00d1c431', 'request_id': 'req-599c5cbd-0d74-4c95-a75d-590f1d55c337'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:49.194865 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-599c5cbd-0d74-4c95-a75d-590f1d55c337 admin admin] The token cache is being invalidate because project cbc73a3a9d0445b785c05e9e00d1c431 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:49.195189 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-599c5cbd-0d74-4c95-a75d-590f1d55c337 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:49.195977 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-599c5cbd-0d74-4c95-a75d-590f1d55c337 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:49.199639 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 944/1879] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:48 2022] DELETE /identity/v3/projects/cbc73a3a9d0445b785c05e9e00d1c431 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:49.205350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb00f74-37b4-4d45-ad46-ab05e5c6d372 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:49.205934 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb00f74-37b4-4d45-ad46-ab05e5c6d372 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:49.206122 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb00f74-37b4-4d45-ad46-ab05e5c6d372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:49.227098 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-cdb00f74-37b4-4d45-ad46-ab05e5c6d372 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:49.228603 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-cdb00f74-37b4-4d45-ad46-ab05e5c6d372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:49.300278 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 936/1880] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:49.309847 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1241104-944d-4cdc-96fa-278ee51e00ba None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:49.310229 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1241104-944d-4cdc-96fa-278ee51e00ba None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:49.310579 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1241104-944d-4cdc-96fa-278ee51e00ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:49.330646 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-f1241104-944d-4cdc-96fa-278ee51e00ba None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:49.331864 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-f1241104-944d-4cdc-96fa-278ee51e00ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:49.344966 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 945/1881] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:49.346434 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb00f74-37b4-4d45-ad46-ab05e5c6d372 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:49.357048 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb00f74-37b4-4d45-ad46-ab05e5c6d372 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:49.362160 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39ec78-79ea-4756-8d7a-54291b312541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:49.362822 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39ec78-79ea-4756-8d7a-54291b312541 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:49.363100 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39ec78-79ea-4756-8d7a-54291b312541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:49.363370 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39ec78-79ea-4756-8d7a-54291b312541 admin admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:49.364100 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc39ec78-79ea-4756-8d7a-54291b312541 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:16:49.364400 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc39ec78-79ea-4756-8d7a-54291b312541 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:49.366030 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc39ec78-79ea-4756-8d7a-54291b312541 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:49.379385 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 937/1882] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:16:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:16:49.384947 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1241104-944d-4cdc-96fa-278ee51e00ba None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:49.396848 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1241104-944d-4cdc-96fa-278ee51e00ba None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:49.404683 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a119a3-87d7-4ca4-b74a-21596118a122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:49.405919 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a119a3-87d7-4ca4-b74a-21596118a122 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:49.405919 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a119a3-87d7-4ca4-b74a-21596118a122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:49.406090 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a119a3-87d7-4ca4-b74a-21596118a122 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:49.406811 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a119a3-87d7-4ca4-b74a-21596118a122 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:49.409037 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a119a3-87d7-4ca4-b74a-21596118a122 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:49.453848 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 946/1883] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:49 2022] POST /identity/v3/projects => generated 379 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:49.455369 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39ec78-79ea-4756-8d7a-54291b312541 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:49.466558 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39ec78-79ea-4756-8d7a-54291b312541 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:49.528581 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595c68ee-ccf6-4bc8-bda8-05b8bf537c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:49.529201 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c68ee-ccf6-4bc8-bda8-05b8bf537c30 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:49.529480 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c68ee-ccf6-4bc8-bda8-05b8bf537c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:49.529747 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c68ee-ccf6-4bc8-bda8-05b8bf537c30 admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:49.530458 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595c68ee-ccf6-4bc8-bda8-05b8bf537c30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:49.533051 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595c68ee-ccf6-4bc8-bda8-05b8bf537c30 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:49.567737 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 938/1884] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:49 2022] POST /identity/v3/users => generated 290 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:16:49.572708 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a119a3-87d7-4ca4-b74a-21596118a122 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:49.583736 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a119a3-87d7-4ca4-b74a-21596118a122 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:49.589696 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42619ec8-3a58-406f-85dc-c3edb339df8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:49.590762 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42619ec8-3a58-406f-85dc-c3edb339df8b admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:49.591002 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42619ec8-3a58-406f-85dc-c3edb339df8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:49.591264 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42619ec8-3a58-406f-85dc-c3edb339df8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:49.592192 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42619ec8-3a58-406f-85dc-c3edb339df8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:49.594148 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42619ec8-3a58-406f-85dc-c3edb339df8b admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:49.611059 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 947/1885] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:49.615816 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595c68ee-ccf6-4bc8-bda8-05b8bf537c30 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:49.627834 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595c68ee-ccf6-4bc8-bda8-05b8bf537c30 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:49.635153 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f9914-7832-4961-95a7-4117ddf12d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:49.635153 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f9914-7832-4961-95a7-4117ddf12d5b admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:49.635488 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f9914-7832-4961-95a7-4117ddf12d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:49.635696 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f9914-7832-4961-95a7-4117ddf12d5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:49.636520 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518f9914-7832-4961-95a7-4117ddf12d5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:49.638651 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518f9914-7832-4961-95a7-4117ddf12d5b admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:49.651249 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 939/1886] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:49.655425 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42619ec8-3a58-406f-85dc-c3edb339df8b admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:49.666837 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42619ec8-3a58-406f-85dc-c3edb339df8b admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:49.674484 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76621b3f-a24c-47b1-bbe3-62e0eaa7539a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:49.675461 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76621b3f-a24c-47b1-bbe3-62e0eaa7539a admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:49.675744 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76621b3f-a24c-47b1-bbe3-62e0eaa7539a admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:49.676021 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76621b3f-a24c-47b1-bbe3-62e0eaa7539a admin admin] PATH_INFO: `/v3/projects/604c9da44965474fb182409ff5aec20e/users/6fb8de51cff14b858bd99f753b3dd88a/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:49.695145 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76621b3f-a24c-47b1-bbe3-62e0eaa7539a admin admin] RBAC: Authorizing `identity:create_grant(project_id=604c9da44965474fb182409ff5aec20e, user_id=6fb8de51cff14b858bd99f753b3dd88a, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:49.698720 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76621b3f-a24c-47b1-bbe3-62e0eaa7539a admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:49.716755 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 948/1887] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:49 2022] PUT /identity/v3/projects/604c9da44965474fb182409ff5aec20e/users/6fb8de51cff14b858bd99f753b3dd88a/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:49.727123 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba7f5f-d28e-4629-bdc9-d1bb82ca7458 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:49.727446 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba7f5f-d28e-4629-bdc9-d1bb82ca7458 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:49.727861 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba7f5f-d28e-4629-bdc9-d1bb82ca7458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:49.747561 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:49.757990 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-3aba7f5f-d28e-4629-bdc9-d1bb82ca7458 None None] MFA Rules not processed for user `6fb8de51cff14b858bd99f753b3dd88a`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:49.759507 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-3aba7f5f-d28e-4629-bdc9-d1bb82ca7458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:49.827636 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 940/1888] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:16:49.832042 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76621b3f-a24c-47b1-bbe3-62e0eaa7539a admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:49.843733 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:49.908712 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76621b3f-a24c-47b1-bbe3-62e0eaa7539a admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:49.913982 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf4376-d546-448f-8726-2bd90ec7d733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:49.914665 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf4376-d546-448f-8726-2bd90ec7d733 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:49.914954 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf4376-d546-448f-8726-2bd90ec7d733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:49.915545 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf4376-d546-448f-8726-2bd90ec7d733 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:49.916290 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbf4376-d546-448f-8726-2bd90ec7d733 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:49.918423 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbf4376-d546-448f-8726-2bd90ec7d733 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:49.983225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 949/1889] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:49 2022] POST /identity/v3/projects => generated 379 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:49.983989 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba7f5f-d28e-4629-bdc9-d1bb82ca7458 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:50.001984 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba7f5f-d28e-4629-bdc9-d1bb82ca7458 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:50.109828 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f703199-7ba7-4577-b922-d9267c666953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:50.111599 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f703199-7ba7-4577-b922-d9267c666953 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:50.112096 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f703199-7ba7-4577-b922-d9267c666953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:50.112423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f703199-7ba7-4577-b922-d9267c666953 admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:50.113328 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f703199-7ba7-4577-b922-d9267c666953 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:50.115960 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f703199-7ba7-4577-b922-d9267c666953 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:50.159643 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 941/1890] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:49 2022] POST /identity/v3/users => generated 296 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:16:50.162040 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf4376-d546-448f-8726-2bd90ec7d733 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:50.180031 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf4376-d546-448f-8726-2bd90ec7d733 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:50.202812 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8541ad7-ef03-4c6c-a60e-37541f76db65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:50.202812 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8541ad7-ef03-4c6c-a60e-37541f76db65 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:50.205326 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8541ad7-ef03-4c6c-a60e-37541f76db65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:50.205326 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8541ad7-ef03-4c6c-a60e-37541f76db65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:50.205326 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8541ad7-ef03-4c6c-a60e-37541f76db65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:50.224901 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8541ad7-ef03-4c6c-a60e-37541f76db65 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:50.255640 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 950/1891] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:50.256029 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f703199-7ba7-4577-b922-d9267c666953 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:50.272122 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f703199-7ba7-4577-b922-d9267c666953 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:50.278213 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b0ff4-4525-4d6d-baf1-9ba3e0dd6639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:50.279202 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b0ff4-4525-4d6d-baf1-9ba3e0dd6639 admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:50.279553 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b0ff4-4525-4d6d-baf1-9ba3e0dd6639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:50.279897 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b0ff4-4525-4d6d-baf1-9ba3e0dd6639 admin admin] PATH_INFO: `/v3/projects/01b4d0280c4e401ebd287b2f531819e3/users/82abbe04cac54858aeb99af9a447a2c7/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:50.310246 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6b0ff4-4525-4d6d-baf1-9ba3e0dd6639 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01b4d0280c4e401ebd287b2f531819e3, user_id=82abbe04cac54858aeb99af9a447a2c7, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:50.313367 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6b0ff4-4525-4d6d-baf1-9ba3e0dd6639 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:50.349123 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 942/1892] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:50 2022] PUT /identity/v3/projects/01b4d0280c4e401ebd287b2f531819e3/users/82abbe04cac54858aeb99af9a447a2c7/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:16:50.349585 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8541ad7-ef03-4c6c-a60e-37541f76db65 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:50.365801 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8541ad7-ef03-4c6c-a60e-37541f76db65 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:50.493358 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4629aa-293f-47f8-8cff-982cf0666ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:50.494161 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4629aa-293f-47f8-8cff-982cf0666ff6 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:50.494990 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4629aa-293f-47f8-8cff-982cf0666ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:50.494990 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4629aa-293f-47f8-8cff-982cf0666ff6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:50.495797 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4629aa-293f-47f8-8cff-982cf0666ff6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:50.497429 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4629aa-293f-47f8-8cff-982cf0666ff6 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:50.510544 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 951/1893] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:50.514683 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b0ff4-4525-4d6d-baf1-9ba3e0dd6639 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:50.526411 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b0ff4-4525-4d6d-baf1-9ba3e0dd6639 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:50.532886 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e740371f-4d2f-4b8d-88ae-88509adb87df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:50.533996 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740371f-4d2f-4b8d-88ae-88509adb87df admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:50.534528 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740371f-4d2f-4b8d-88ae-88509adb87df admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:50.534810 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740371f-4d2f-4b8d-88ae-88509adb87df admin admin] PATH_INFO: `/v3/projects/01b4d0280c4e401ebd287b2f531819e3/users/82abbe04cac54858aeb99af9a447a2c7/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:50.536829 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e740371f-4d2f-4b8d-88ae-88509adb87df admin admin] RBAC: Authorizing `identity:create_grant(project_id=01b4d0280c4e401ebd287b2f531819e3, user_id=82abbe04cac54858aeb99af9a447a2c7, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:50.539915 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e740371f-4d2f-4b8d-88ae-88509adb87df admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:50.553936 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 943/1894] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:50 2022] PUT /identity/v3/projects/01b4d0280c4e401ebd287b2f531819e3/users/82abbe04cac54858aeb99af9a447a2c7/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:50.564386 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f59ae-2eb0-46e2-832e-d7a6eb84f3a7 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:50.564668 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f59ae-2eb0-46e2-832e-d7a6eb84f3a7 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:50.564968 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041f59ae-2eb0-46e2-832e-d7a6eb84f3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:50.594395 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-041f59ae-2eb0-46e2-832e-d7a6eb84f3a7 None None] MFA Rules not processed for user `82abbe04cac54858aeb99af9a447a2c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:50.595673 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-041f59ae-2eb0-46e2-832e-d7a6eb84f3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:50.690341 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 952/1895] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:50 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:50.701911 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e740371f-4d2f-4b8d-88ae-88509adb87df admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:50.703647 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-e740371f-4d2f-4b8d-88ae-88509adb87df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:50.780893 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e740371f-4d2f-4b8d-88ae-88509adb87df admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:50.786741 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbcff02-c24e-4c95-add5-0e04a4ac4e17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:50.787700 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbcff02-c24e-4c95-add5-0e04a4ac4e17 service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:50.788081 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbcff02-c24e-4c95-add5-0e04a4ac4e17 service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:50.788652 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbcff02-c24e-4c95-add5-0e04a4ac4e17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:50.796487 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:50.804360 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbcff02-c24e-4c95-add5-0e04a4ac4e17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:50.807919 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbcff02-c24e-4c95-add5-0e04a4ac4e17 service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:50.827319 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 944/1896] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:50 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:51.341984 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f59ae-2eb0-46e2-832e-d7a6eb84f3a7 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:51.347542 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:51.427913 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041f59ae-2eb0-46e2-832e-d7a6eb84f3a7 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:51.441318 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b2a77-23c0-4401-a6a6-e9874ed2a761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:51.444168 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b2a77-23c0-4401-a6a6-e9874ed2a761 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:51.445008 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b2a77-23c0-4401-a6a6-e9874ed2a761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:51.445008 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b2a77-23c0-4401-a6a6-e9874ed2a761 admin admin] PATH_INFO: `/v3/users/6fb8de51cff14b858bd99f753b3dd88a` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:51.449696 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741b2a77-23c0-4401-a6a6-e9874ed2a761 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fb8de51cff14b858bd99f753b3dd88a)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:51.452021 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741b2a77-23c0-4401-a6a6-e9874ed2a761 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:51.563247 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-741b2a77-23c0-4401-a6a6-e9874ed2a761 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fb8de51cff14b858bd99f753b3dd88a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:51.574160 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-741b2a77-23c0-4401-a6a6-e9874ed2a761 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fb8de51cff14b858bd99f753b3dd88a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:51.574741 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-741b2a77-23c0-4401-a6a6-e9874ed2a761 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fb8de51cff14b858bd99f753b3dd88a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:51.576048 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-741b2a77-23c0-4401-a6a6-e9874ed2a761 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fb8de51cff14b858bd99f753b3dd88a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:51.591125 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-741b2a77-23c0-4401-a6a6-e9874ed2a761 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fb8de51cff14b858bd99f753b3dd88a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:51.630581 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 953/1897] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:16:51 2022] DELETE /identity/v3/users/6fb8de51cff14b858bd99f753b3dd88a => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:51.636217 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ba020-a005-4bf3-8374-2b1cd8909df6 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:51.636812 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ba020-a005-4bf3-8374-2b1cd8909df6 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:51.636812 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ba020-a005-4bf3-8374-2b1cd8909df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:51.662109 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-094ba020-a005-4bf3-8374-2b1cd8909df6 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:51.663854 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-094ba020-a005-4bf3-8374-2b1cd8909df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:51.762348 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 945/1898] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:51.802021 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b2a77-23c0-4401-a6a6-e9874ed2a761 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:51.803645 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-741b2a77-23c0-4401-a6a6-e9874ed2a761 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:51.928388 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b2a77-23c0-4401-a6a6-e9874ed2a761 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:51.933660 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e5d53b-e2d1-4257-bfd8-6963836b59d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:51.934808 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5d53b-e2d1-4257-bfd8-6963836b59d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:51.935321 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5d53b-e2d1-4257-bfd8-6963836b59d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:51.935786 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5d53b-e2d1-4257-bfd8-6963836b59d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:51.950882 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e5d53b-e2d1-4257-bfd8-6963836b59d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:51.952546 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e5d53b-e2d1-4257-bfd8-6963836b59d9 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:51.964986 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 954/1899] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:52.654485 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ba020-a005-4bf3-8374-2b1cd8909df6 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:52.656068 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-094ba020-a005-4bf3-8374-2b1cd8909df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:52.670271 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ba020-a005-4bf3-8374-2b1cd8909df6 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:52.678510 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d30e6-b020-437d-90b7-3c4af6ded39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:52.679723 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d30e6-b020-437d-90b7-3c4af6ded39e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:52.680217 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d30e6-b020-437d-90b7-3c4af6ded39e admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:52.680807 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d30e6-b020-437d-90b7-3c4af6ded39e admin admin] PATH_INFO: `/v3/projects/604c9da44965474fb182409ff5aec20e` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:52.682978 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d30e6-b020-437d-90b7-3c4af6ded39e admin admin] RBAC: Authorizing `identity:delete_project(project_id=604c9da44965474fb182409ff5aec20e)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:52.685981 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d30e6-b020-437d-90b7-3c4af6ded39e admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:52.758130 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-622d30e6-b020-437d-90b7-3c4af6ded39e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '604c9da44965474fb182409ff5aec20e', 'request_id': 'req-622d30e6-b020-437d-90b7-3c4af6ded39e'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:52.765538 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-622d30e6-b020-437d-90b7-3c4af6ded39e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '604c9da44965474fb182409ff5aec20e', 'request_id': 'req-622d30e6-b020-437d-90b7-3c4af6ded39e'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:52.776378 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-622d30e6-b020-437d-90b7-3c4af6ded39e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '604c9da44965474fb182409ff5aec20e', 'request_id': 'req-622d30e6-b020-437d-90b7-3c4af6ded39e'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:52.791936 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-622d30e6-b020-437d-90b7-3c4af6ded39e admin admin] The token cache is being invalidate because project 604c9da44965474fb182409ff5aec20e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:52.792141 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-622d30e6-b020-437d-90b7-3c4af6ded39e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:52.792796 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-622d30e6-b020-437d-90b7-3c4af6ded39e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:52.794400 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 946/1900] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:52 2022] DELETE /identity/v3/projects/604c9da44965474fb182409ff5aec20e => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:52.798397 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e5d53b-e2d1-4257-bfd8-6963836b59d9 service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:52.809926 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e5d53b-e2d1-4257-bfd8-6963836b59d9 service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:52.842320 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:53.047545 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9844a03-711b-4644-b7e0-094a1ed2eac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:53.048696 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9844a03-711b-4644-b7e0-094a1ed2eac8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:53.048973 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9844a03-711b-4644-b7e0-094a1ed2eac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:53.049366 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9844a03-711b-4644-b7e0-094a1ed2eac8 admin admin] PATH_INFO: `/v3/users/82abbe04cac54858aeb99af9a447a2c7` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:53.050695 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9844a03-711b-4644-b7e0-094a1ed2eac8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82abbe04cac54858aeb99af9a447a2c7)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:53.052728 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9844a03-711b-4644-b7e0-094a1ed2eac8 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:53.117099 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a9844a03-711b-4644-b7e0-094a1ed2eac8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82abbe04cac54858aeb99af9a447a2c7'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:53.131440 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a9844a03-711b-4644-b7e0-094a1ed2eac8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82abbe04cac54858aeb99af9a447a2c7'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:53.131440 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a9844a03-711b-4644-b7e0-094a1ed2eac8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82abbe04cac54858aeb99af9a447a2c7'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:53.131440 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a9844a03-711b-4644-b7e0-094a1ed2eac8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82abbe04cac54858aeb99af9a447a2c7'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:53.142563 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a9844a03-711b-4644-b7e0-094a1ed2eac8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82abbe04cac54858aeb99af9a447a2c7'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:53.165686 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 955/1901] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:16:52 2022] DELETE /identity/v3/users/82abbe04cac54858aeb99af9a447a2c7 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:53.783281 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d30e6-b020-437d-90b7-3c4af6ded39e admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:53.784956 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-622d30e6-b020-437d-90b7-3c4af6ded39e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:53.800459 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d30e6-b020-437d-90b7-3c4af6ded39e admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:53.869220 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c4737-af1a-445b-844c-c2b4da46581c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:53.869860 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c4737-af1a-445b-844c-c2b4da46581c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:53.870140 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c4737-af1a-445b-844c-c2b4da46581c admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:53.870416 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c4737-af1a-445b-844c-c2b4da46581c admin admin] PATH_INFO: `/v3/projects/01b4d0280c4e401ebd287b2f531819e3` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:53.871673 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9c4737-af1a-445b-844c-c2b4da46581c admin admin] RBAC: Authorizing `identity:delete_project(project_id=01b4d0280c4e401ebd287b2f531819e3)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:53.873770 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9c4737-af1a-445b-844c-c2b4da46581c admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:53.899245 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:53.930947 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-7d9c4737-af1a-445b-844c-c2b4da46581c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01b4d0280c4e401ebd287b2f531819e3', 'request_id': 'req-7d9c4737-af1a-445b-844c-c2b4da46581c'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:53.937941 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-7d9c4737-af1a-445b-844c-c2b4da46581c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01b4d0280c4e401ebd287b2f531819e3', 'request_id': 'req-7d9c4737-af1a-445b-844c-c2b4da46581c'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:53.947326 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-7d9c4737-af1a-445b-844c-c2b4da46581c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01b4d0280c4e401ebd287b2f531819e3', 'request_id': 'req-7d9c4737-af1a-445b-844c-c2b4da46581c'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:53.961785 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-7d9c4737-af1a-445b-844c-c2b4da46581c admin admin] The token cache is being invalidate because project 01b4d0280c4e401ebd287b2f531819e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:16:53.962308 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-7d9c4737-af1a-445b-844c-c2b4da46581c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:53.963510 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-7d9c4737-af1a-445b-844c-c2b4da46581c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:16:53.965252 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 947/1902] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:16:53 2022] DELETE /identity/v3/projects/01b4d0280c4e401ebd287b2f531819e3 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:53.972375 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd7f96-d856-414e-a3dc-52488d78862e None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:53.972890 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd7f96-d856-414e-a3dc-52488d78862e None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:53.973159 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfd7f96-d856-414e-a3dc-52488d78862e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:53.992391 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-ecfd7f96-d856-414e-a3dc-52488d78862e None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:53.993803 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ecfd7f96-d856-414e-a3dc-52488d78862e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:54.076035 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 956/1903] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:54.085169 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5202c7d-c048-4cc2-b99c-fbef8b5e226e None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:54.085459 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5202c7d-c048-4cc2-b99c-fbef8b5e226e None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:54.085730 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5202c7d-c048-4cc2-b99c-fbef8b5e226e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:54.106379 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-c5202c7d-c048-4cc2-b99c-fbef8b5e226e None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:54.107660 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-c5202c7d-c048-4cc2-b99c-fbef8b5e226e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:54.116814 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 948/1904] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:54.121688 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd7f96-d856-414e-a3dc-52488d78862e None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:54.131688 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfd7f96-d856-414e-a3dc-52488d78862e None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:54.138185 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a110bd-57cc-448e-b8b6-b9a533b78d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:54.139214 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a110bd-57cc-448e-b8b6-b9a533b78d52 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:54.139371 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a110bd-57cc-448e-b8b6-b9a533b78d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:54.139642 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a110bd-57cc-448e-b8b6-b9a533b78d52 admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:54.140413 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a110bd-57cc-448e-b8b6-b9a533b78d52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:16:54.140718 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a110bd-57cc-448e-b8b6-b9a533b78d52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:54.142442 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a110bd-57cc-448e-b8b6-b9a533b78d52 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:54.155801 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 957/1905] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:16:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:16:54.160474 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5202c7d-c048-4cc2-b99c-fbef8b5e226e None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:54.170791 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5202c7d-c048-4cc2-b99c-fbef8b5e226e None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:54.182455 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad2c61-c3c8-46c2-88bb-8714449da7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:54.183686 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad2c61-c3c8-46c2-88bb-8714449da7c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:54.183686 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad2c61-c3c8-46c2-88bb-8714449da7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:54.183948 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad2c61-c3c8-46c2-88bb-8714449da7c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:54.184729 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ad2c61-c3c8-46c2-88bb-8714449da7c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:54.186783 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ad2c61-c3c8-46c2-88bb-8714449da7c2 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:54.225837 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 949/1906] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:54 2022] POST /identity/v3/projects => generated 377 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:54.227695 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a110bd-57cc-448e-b8b6-b9a533b78d52 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:54.238939 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a110bd-57cc-448e-b8b6-b9a533b78d52 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:54.280451 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:54.314000 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c24a2-126a-485b-b1ca-6910ac5c8a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:54.314840 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c24a2-126a-485b-b1ca-6910ac5c8a53 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:54.315126 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c24a2-126a-485b-b1ca-6910ac5c8a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:54.315401 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c24a2-126a-485b-b1ca-6910ac5c8a53 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:54.316280 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c24a2-126a-485b-b1ca-6910ac5c8a53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:54.318354 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c24a2-126a-485b-b1ca-6910ac5c8a53 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:54.349672 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 958/1907] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:54 2022] POST /identity/v3/users => generated 289 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:54.351503 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad2c61-c3c8-46c2-88bb-8714449da7c2 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:54.377397 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad2c61-c3c8-46c2-88bb-8714449da7c2 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:54.382916 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a24547-47f3-4c79-b182-3a7155980246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:54.383710 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a24547-47f3-4c79-b182-3a7155980246 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:54.384073 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a24547-47f3-4c79-b182-3a7155980246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:54.384606 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a24547-47f3-4c79-b182-3a7155980246 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:54.385459 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a24547-47f3-4c79-b182-3a7155980246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:54.387183 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a24547-47f3-4c79-b182-3a7155980246 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:54.399597 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 950/1908] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:54 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:54.407185 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c24a2-126a-485b-b1ca-6910ac5c8a53 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:54.421709 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c24a2-126a-485b-b1ca-6910ac5c8a53 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:54.427017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf63fc-a156-443c-8a1f-a455cc975719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:54.428606 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf63fc-a156-443c-8a1f-a455cc975719 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:54.428606 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf63fc-a156-443c-8a1f-a455cc975719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:54.428606 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf63fc-a156-443c-8a1f-a455cc975719 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:54.429077 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cf63fc-a156-443c-8a1f-a455cc975719 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:54.430822 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cf63fc-a156-443c-8a1f-a455cc975719 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:54.446420 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 959/1909] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:54.452396 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a24547-47f3-4c79-b182-3a7155980246 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:54.462165 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a24547-47f3-4c79-b182-3a7155980246 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:54.470344 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec59fc-6044-463e-8123-83427c616388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:54.471068 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec59fc-6044-463e-8123-83427c616388 admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:54.471346 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec59fc-6044-463e-8123-83427c616388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:54.471623 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec59fc-6044-463e-8123-83427c616388 admin admin] PATH_INFO: `/v3/projects/6bfa6be0235e4bbdbb610c1ab68059f7/users/b0c77f6f328d47e9a7eb972cc4053818/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:54.492476 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ec59fc-6044-463e-8123-83427c616388 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bfa6be0235e4bbdbb610c1ab68059f7, user_id=b0c77f6f328d47e9a7eb972cc4053818, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:54.495570 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ec59fc-6044-463e-8123-83427c616388 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:54.521352 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 951/1910] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:54 2022] PUT /identity/v3/projects/6bfa6be0235e4bbdbb610c1ab68059f7/users/b0c77f6f328d47e9a7eb972cc4053818/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:54.527301 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f65ea-90a8-47e2-8e72-d84daa870443 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:54.528121 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f65ea-90a8-47e2-8e72-d84daa870443 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:54.528121 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f65ea-90a8-47e2-8e72-d84daa870443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:54.591920 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-3f6f65ea-90a8-47e2-8e72-d84daa870443 None None] MFA Rules not processed for user `b0c77f6f328d47e9a7eb972cc4053818`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:54.595567 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-3f6f65ea-90a8-47e2-8e72-d84daa870443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:54.801282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 960/1911] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:54.805298 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec59fc-6044-463e-8123-83427c616388 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:54.969341 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec59fc-6044-463e-8123-83427c616388 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:54.976113 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5a62a-84d7-4715-833f-7c1e432cff71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:54.976800 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5a62a-84d7-4715-833f-7c1e432cff71 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:54.977112 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5a62a-84d7-4715-833f-7c1e432cff71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:54.977384 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5a62a-84d7-4715-833f-7c1e432cff71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:54.978428 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f5a62a-84d7-4715-833f-7c1e432cff71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:54.981210 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f5a62a-84d7-4715-833f-7c1e432cff71 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:55.033463 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 952/1912] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:54 2022] POST /identity/v3/projects => generated 375 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:55.038103 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6f65ea-90a8-47e2-8e72-d84daa870443 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:55.056276 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6f65ea-90a8-47e2-8e72-d84daa870443 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:55.138654 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac46363-87c8-43e1-993a-05117455aedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:55.139262 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac46363-87c8-43e1-993a-05117455aedb admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:55.139548 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac46363-87c8-43e1-993a-05117455aedb admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:55.139816 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac46363-87c8-43e1-993a-05117455aedb admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:55.141019 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac46363-87c8-43e1-993a-05117455aedb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:55.142505 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac46363-87c8-43e1-993a-05117455aedb admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:55.168632 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 961/1913] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:55 2022] POST /identity/v3/users => generated 288 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:55.169984 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5a62a-84d7-4715-833f-7c1e432cff71 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:55.181975 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5a62a-84d7-4715-833f-7c1e432cff71 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:55.187333 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7b7b7-a68a-4aaa-9c0e-99b969a9e948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:55.188102 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7b7b7-a68a-4aaa-9c0e-99b969a9e948 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:55.188382 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7b7b7-a68a-4aaa-9c0e-99b969a9e948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:55.188652 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7b7b7-a68a-4aaa-9c0e-99b969a9e948 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:55.189474 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e7b7b7-a68a-4aaa-9c0e-99b969a9e948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:55.191207 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e7b7b7-a68a-4aaa-9c0e-99b969a9e948 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:55.218079 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 953/1914] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:55 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:55.222621 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac46363-87c8-43e1-993a-05117455aedb admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:55.233722 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac46363-87c8-43e1-993a-05117455aedb admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:55.239123 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92514ecb-9533-4239-8e63-309cbe1a30e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:55.239893 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92514ecb-9533-4239-8e63-309cbe1a30e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:55.240275 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92514ecb-9533-4239-8e63-309cbe1a30e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:55.240609 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92514ecb-9533-4239-8e63-309cbe1a30e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:55.241505 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92514ecb-9533-4239-8e63-309cbe1a30e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:55.243341 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92514ecb-9533-4239-8e63-309cbe1a30e6 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:55.255522 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 962/1915] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:55.262991 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7b7b7-a68a-4aaa-9c0e-99b969a9e948 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:55.274846 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7b7b7-a68a-4aaa-9c0e-99b969a9e948 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:55.280323 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25fc57c-5a77-4470-8081-66408f9bacc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:55.281299 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25fc57c-5a77-4470-8081-66408f9bacc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:55.281579 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25fc57c-5a77-4470-8081-66408f9bacc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:55.281854 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25fc57c-5a77-4470-8081-66408f9bacc5 admin admin] PATH_INFO: `/v3/projects/56df7ea73bf64ae3ab7d91a0c6c475d2/users/59232c9123f14b898eec6a2d428b5897/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:55.305436 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25fc57c-5a77-4470-8081-66408f9bacc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56df7ea73bf64ae3ab7d91a0c6c475d2, user_id=59232c9123f14b898eec6a2d428b5897, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:55.309582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25fc57c-5a77-4470-8081-66408f9bacc5 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:55.312789 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:55.329203 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 954/1916] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:55 2022] PUT /identity/v3/projects/56df7ea73bf64ae3ab7d91a0c6c475d2/users/59232c9123f14b898eec6a2d428b5897/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:55.341525 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed7bf3-3620-4560-aeb7-40d161bf0b9f None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:55.341773 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed7bf3-3620-4560-aeb7-40d161bf0b9f None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:55.342087 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed7bf3-3620-4560-aeb7-40d161bf0b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:55.375284 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-ceed7bf3-3620-4560-aeb7-40d161bf0b9f None None] MFA Rules not processed for user `59232c9123f14b898eec6a2d428b5897`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:55.376603 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ceed7bf3-3620-4560-aeb7-40d161bf0b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:55.385951 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:55.447592 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 963/1917] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:16:55.452118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25fc57c-5a77-4470-8081-66408f9bacc5 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:55.518839 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25fc57c-5a77-4470-8081-66408f9bacc5 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:55.524777 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4761a46-935f-4165-a990-48f35468548b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:55.525482 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4761a46-935f-4165-a990-48f35468548b admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:55.525758 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4761a46-935f-4165-a990-48f35468548b admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:55.526033 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4761a46-935f-4165-a990-48f35468548b admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:55.526772 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4761a46-935f-4165-a990-48f35468548b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:55.528690 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4761a46-935f-4165-a990-48f35468548b admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:55.558302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 955/1918] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:16:55 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:55.562632 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed7bf3-3620-4560-aeb7-40d161bf0b9f None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:55.575194 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed7bf3-3620-4560-aeb7-40d161bf0b9f None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:55.650927 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bc929c-86c4-4737-8e48-469e217973e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:55.651972 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc929c-86c4-4737-8e48-469e217973e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:55.652484 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc929c-86c4-4737-8e48-469e217973e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:55.653564 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc929c-86c4-4737-8e48-469e217973e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:55.654377 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bc929c-86c4-4737-8e48-469e217973e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:55.656735 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bc929c-86c4-4737-8e48-469e217973e9 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:55.681083 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 964/1919] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:16:55 2022] POST /identity/v3/users => generated 295 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:16:55.685332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4761a46-935f-4165-a990-48f35468548b admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:55.698929 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4761a46-935f-4165-a990-48f35468548b admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:55.704513 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29eb23-a08e-471e-a2cf-9e247f633f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:55.705273 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29eb23-a08e-471e-a2cf-9e247f633f14 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:55.705557 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29eb23-a08e-471e-a2cf-9e247f633f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:55.705833 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29eb23-a08e-471e-a2cf-9e247f633f14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:55.706578 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c29eb23-a08e-471e-a2cf-9e247f633f14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:55.708212 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c29eb23-a08e-471e-a2cf-9e247f633f14 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:55.721687 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 956/1920] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:55.725453 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bc929c-86c4-4737-8e48-469e217973e9 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:55.738218 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bc929c-86c4-4737-8e48-469e217973e9 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:55.747230 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f0bfb-e38e-46d8-a05c-780f599b51f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:55.748193 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f0bfb-e38e-46d8-a05c-780f599b51f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:55.748462 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f0bfb-e38e-46d8-a05c-780f599b51f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:55.748731 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f0bfb-e38e-46d8-a05c-780f599b51f3 admin admin] PATH_INFO: `/v3/projects/b129240935544b9487c2e2fdb2414353/users/63c66af4436a482182ed082aacf60fb1/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:55.771817 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70f0bfb-e38e-46d8-a05c-780f599b51f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b129240935544b9487c2e2fdb2414353, user_id=63c66af4436a482182ed082aacf60fb1, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:55.777486 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70f0bfb-e38e-46d8-a05c-780f599b51f3 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:55.813021 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 965/1921] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:55 2022] PUT /identity/v3/projects/b129240935544b9487c2e2fdb2414353/users/63c66af4436a482182ed082aacf60fb1/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:16:55.814430 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29eb23-a08e-471e-a2cf-9e247f633f14 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:55.830301 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29eb23-a08e-471e-a2cf-9e247f633f14 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:55.999258 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c079eb3-86be-47e6-b00b-e3ee9359ede2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:56.000014 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c079eb3-86be-47e6-b00b-e3ee9359ede2 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:56.000393 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c079eb3-86be-47e6-b00b-e3ee9359ede2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:56.000762 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c079eb3-86be-47e6-b00b-e3ee9359ede2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:56.003354 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c079eb3-86be-47e6-b00b-e3ee9359ede2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:56.005386 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c079eb3-86be-47e6-b00b-e3ee9359ede2 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:56.035117 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 957/1922] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:16:55 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:16:56.035237 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f0bfb-e38e-46d8-a05c-780f599b51f3 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:56.059863 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f0bfb-e38e-46d8-a05c-780f599b51f3 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:56.070602 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f051f89-ea93-47c2-930f-9086738ca21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:56.072201 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f051f89-ea93-47c2-930f-9086738ca21f admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:56.072579 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f051f89-ea93-47c2-930f-9086738ca21f admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:56.072970 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f051f89-ea93-47c2-930f-9086738ca21f admin admin] PATH_INFO: `/v3/projects/b129240935544b9487c2e2fdb2414353/users/63c66af4436a482182ed082aacf60fb1/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:56.075384 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f051f89-ea93-47c2-930f-9086738ca21f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b129240935544b9487c2e2fdb2414353, user_id=63c66af4436a482182ed082aacf60fb1, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:56.083464 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f051f89-ea93-47c2-930f-9086738ca21f admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:56.110102 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e84bfd-d162-4c24-af12-5441babc962e None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:56.110689 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e84bfd-d162-4c24-af12-5441babc962e None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:56.111634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e84bfd-d162-4c24-af12-5441babc962e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:56.113241 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 966/1923] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:16:56 2022] PUT /identity/v3/projects/b129240935544b9487c2e2fdb2414353/users/63c66af4436a482182ed082aacf60fb1/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:16:56.150903 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-d7e84bfd-d162-4c24-af12-5441babc962e None None] MFA Rules not processed for user `63c66af4436a482182ed082aacf60fb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:16:56.152379 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-d7e84bfd-d162-4c24-af12-5441babc962e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:56.240418 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:56.288117 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 958/1924] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:16:56 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:16:56.322781 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f051f89-ea93-47c2-930f-9086738ca21f admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:56.329911 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-4f051f89-ea93-47c2-930f-9086738ca21f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:16:56.438772 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f051f89-ea93-47c2-930f-9086738ca21f admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:56.447819 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ade48-a2cc-4794-8333-df7fca091db2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:56.448614 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ade48-a2cc-4794-8333-df7fca091db2 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:56.449006 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ade48-a2cc-4794-8333-df7fca091db2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:56.449398 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ade48-a2cc-4794-8333-df7fca091db2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:56.474105 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:16:56.557932 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309ade48-a2cc-4794-8333-df7fca091db2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:56.559988 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309ade48-a2cc-4794-8333-df7fca091db2 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:56.575387 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 967/1925] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:56 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:58.309860 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e84bfd-d162-4c24-af12-5441babc962e None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:58.328325 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e84bfd-d162-4c24-af12-5441babc962e None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:58.342734 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754b7dde-b1d4-40b2-86c1-50b09a472f34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:58.344456 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754b7dde-b1d4-40b2-86c1-50b09a472f34 service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:58.344960 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754b7dde-b1d4-40b2-86c1-50b09a472f34 service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:58.345600 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754b7dde-b1d4-40b2-86c1-50b09a472f34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:58.426550 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754b7dde-b1d4-40b2-86c1-50b09a472f34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:58.428329 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754b7dde-b1d4-40b2-86c1-50b09a472f34 service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:58.438617 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 959/1926] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:16:59.446413 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ade48-a2cc-4794-8333-df7fca091db2 service cinder] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:16:59.459047 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ade48-a2cc-4794-8333-df7fca091db2 service cinder] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:16:59.465391 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75719f67-d150-44b3-ae82-50a7794c8a64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:16:59.465855 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75719f67-d150-44b3-ae82-50a7794c8a64 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:16:59.466086 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75719f67-d150-44b3-ae82-50a7794c8a64 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:16:59.473740 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75719f67-d150-44b3-ae82-50a7794c8a64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:16:59.488043 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75719f67-d150-44b3-ae82-50a7794c8a64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:16:59.489888 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75719f67-d150-44b3-ae82-50a7794c8a64 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:16:59.501944 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 968/1927] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:16:59 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:00.458992 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8317db-858f-44f3-9575-05d46fb1d3ea None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:00.459585 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8317db-858f-44f3-9575-05d46fb1d3ea None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:00.459585 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8317db-858f-44f3-9575-05d46fb1d3ea None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:00.461174 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 960/1928] 10.222.0.13 () {54 vars in 921 bytes} [Mon Jun 27 21:17:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:17:00.482970 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e0bc0-5334-4057-a196-8947183f5289 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:00.483580 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e0bc0-5334-4057-a196-8947183f5289 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:00.483580 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e0bc0-5334-4057-a196-8947183f5289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:00.507552 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-1a8e0bc0-5334-4057-a196-8947183f5289 None None] MFA Rules not processed for user `0437c45d72cd4a38b9d48ee6f50946fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:00.509315 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-1a8e0bc0-5334-4057-a196-8947183f5289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:00.594977 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 969/1929] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Jun 27 21:17:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:00.620144 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8317db-858f-44f3-9575-05d46fb1d3ea None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:00.622076 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-7c8317db-858f-44f3-9575-05d46fb1d3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:00.640332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8317db-858f-44f3-9575-05d46fb1d3ea None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:00.645473 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636820d2-8e89-48b2-80a3-df7413c17292 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0437c45d72cd4a38b9d48ee6f50946fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:00.646219 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636820d2-8e89-48b2-80a3-df7413c17292 service placement] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:00.646567 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636820d2-8e89-48b2-80a3-df7413c17292 service placement] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:00.646913 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636820d2-8e89-48b2-80a3-df7413c17292 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:00.657752 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636820d2-8e89-48b2-80a3-df7413c17292 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:00.659471 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636820d2-8e89-48b2-80a3-df7413c17292 service placement] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:00.671898 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 961/1930] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:01.646133 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e0bc0-5334-4057-a196-8947183f5289 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:01.648607 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-1a8e0bc0-5334-4057-a196-8947183f5289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:01.755576 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e0bc0-5334-4057-a196-8947183f5289 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:01.762437 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb9304a-da9f-4068-8df6-5a64b45d46b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:01.763811 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb9304a-da9f-4068-8df6-5a64b45d46b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:01.764136 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb9304a-da9f-4068-8df6-5a64b45d46b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:01.764431 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb9304a-da9f-4068-8df6-5a64b45d46b6 admin admin] PATH_INFO: `/v3/users/7b98f3b3cf174d678af5d8c9f2949018` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:01.766316 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb9304a-da9f-4068-8df6-5a64b45d46b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b98f3b3cf174d678af5d8c9f2949018)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:01.767824 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb9304a-da9f-4068-8df6-5a64b45d46b6 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:01.838161 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:01.844569 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-4eb9304a-da9f-4068-8df6-5a64b45d46b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b98f3b3cf174d678af5d8c9f2949018'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:01.854362 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-4eb9304a-da9f-4068-8df6-5a64b45d46b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b98f3b3cf174d678af5d8c9f2949018'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:01.855217 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-4eb9304a-da9f-4068-8df6-5a64b45d46b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b98f3b3cf174d678af5d8c9f2949018'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:01.856032 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-4eb9304a-da9f-4068-8df6-5a64b45d46b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b98f3b3cf174d678af5d8c9f2949018'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:01.866122 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-4eb9304a-da9f-4068-8df6-5a64b45d46b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b98f3b3cf174d678af5d8c9f2949018'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:01.887063 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 970/1931] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:17:01 2022] DELETE /identity/v3/users/7b98f3b3cf174d678af5d8c9f2949018 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:02.082808 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636820d2-8e89-48b2-80a3-df7413c17292 service placement] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:02.115862 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636820d2-8e89-48b2-80a3-df7413c17292 service placement] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:02.192704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234f4326-53ce-49cc-aa29-0d5fd164a537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:02.193635 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f4326-53ce-49cc-aa29-0d5fd164a537 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:02.193914 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f4326-53ce-49cc-aa29-0d5fd164a537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:02.194362 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f4326-53ce-49cc-aa29-0d5fd164a537 admin admin] PATH_INFO: `/v3/projects/7fb32a0153c1491bb10c2e798f1fcf46` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:02.196264 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234f4326-53ce-49cc-aa29-0d5fd164a537 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fb32a0153c1491bb10c2e798f1fcf46)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:02.198819 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234f4326-53ce-49cc-aa29-0d5fd164a537 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:02.271180 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-234f4326-53ce-49cc-aa29-0d5fd164a537 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fb32a0153c1491bb10c2e798f1fcf46', 'request_id': 'req-234f4326-53ce-49cc-aa29-0d5fd164a537'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:02.280870 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-234f4326-53ce-49cc-aa29-0d5fd164a537 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fb32a0153c1491bb10c2e798f1fcf46', 'request_id': 'req-234f4326-53ce-49cc-aa29-0d5fd164a537'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:02.291240 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-234f4326-53ce-49cc-aa29-0d5fd164a537 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fb32a0153c1491bb10c2e798f1fcf46', 'request_id': 'req-234f4326-53ce-49cc-aa29-0d5fd164a537'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:02.304494 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-234f4326-53ce-49cc-aa29-0d5fd164a537 admin admin] The token cache is being invalidate because project 7fb32a0153c1491bb10c2e798f1fcf46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:17:02.304788 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-234f4326-53ce-49cc-aa29-0d5fd164a537 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:02.305544 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-234f4326-53ce-49cc-aa29-0d5fd164a537 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:02.307182 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 962/1932] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:17:02 2022] DELETE /identity/v3/projects/7fb32a0153c1491bb10c2e798f1fcf46 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:02.313048 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb9304a-da9f-4068-8df6-5a64b45d46b6 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:02.314579 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-4eb9304a-da9f-4068-8df6-5a64b45d46b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:02.340276 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb9304a-da9f-4068-8df6-5a64b45d46b6 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:02.418837 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b5b3a-c35f-43ec-9aa4-468695be2e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:02.419853 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b5b3a-c35f-43ec-9aa4-468695be2e93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:02.420210 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b5b3a-c35f-43ec-9aa4-468695be2e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:02.420513 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b5b3a-c35f-43ec-9aa4-468695be2e93 admin admin] PATH_INFO: `/v3/users/2bbf85747f624ab6bfb4b4eec90c3ca8` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:02.421887 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154b5b3a-c35f-43ec-9aa4-468695be2e93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bbf85747f624ab6bfb4b4eec90c3ca8)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:02.424085 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154b5b3a-c35f-43ec-9aa4-468695be2e93 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:02.497875 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-154b5b3a-c35f-43ec-9aa4-468695be2e93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bbf85747f624ab6bfb4b4eec90c3ca8'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:02.526868 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-154b5b3a-c35f-43ec-9aa4-468695be2e93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bbf85747f624ab6bfb4b4eec90c3ca8'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:02.526868 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-154b5b3a-c35f-43ec-9aa4-468695be2e93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bbf85747f624ab6bfb4b4eec90c3ca8'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:02.526868 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-154b5b3a-c35f-43ec-9aa4-468695be2e93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bbf85747f624ab6bfb4b4eec90c3ca8'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:02.540191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-154b5b3a-c35f-43ec-9aa4-468695be2e93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bbf85747f624ab6bfb4b4eec90c3ca8'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:02.563022 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 971/1933] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:17:02 2022] DELETE /identity/v3/users/2bbf85747f624ab6bfb4b4eec90c3ca8 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:02.783870 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234f4326-53ce-49cc-aa29-0d5fd164a537 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:02.785521 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-234f4326-53ce-49cc-aa29-0d5fd164a537 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:02.803090 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:02.808438 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234f4326-53ce-49cc-aa29-0d5fd164a537 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:02.901131 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709fbef1-b1ba-4aa9-a718-3aaf518b719d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:02.902155 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709fbef1-b1ba-4aa9-a718-3aaf518b719d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:02.902439 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709fbef1-b1ba-4aa9-a718-3aaf518b719d admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:02.902719 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709fbef1-b1ba-4aa9-a718-3aaf518b719d admin admin] PATH_INFO: `/v3/projects/f044bbb8bb2d4af9ae27d3b849e3a369` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:02.904039 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709fbef1-b1ba-4aa9-a718-3aaf518b719d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f044bbb8bb2d4af9ae27d3b849e3a369)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:02.906407 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709fbef1-b1ba-4aa9-a718-3aaf518b719d admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:02.988009 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-709fbef1-b1ba-4aa9-a718-3aaf518b719d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f044bbb8bb2d4af9ae27d3b849e3a369', 'request_id': 'req-709fbef1-b1ba-4aa9-a718-3aaf518b719d'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:02.995391 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-709fbef1-b1ba-4aa9-a718-3aaf518b719d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f044bbb8bb2d4af9ae27d3b849e3a369', 'request_id': 'req-709fbef1-b1ba-4aa9-a718-3aaf518b719d'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:03.003571 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-709fbef1-b1ba-4aa9-a718-3aaf518b719d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f044bbb8bb2d4af9ae27d3b849e3a369', 'request_id': 'req-709fbef1-b1ba-4aa9-a718-3aaf518b719d'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:03.014096 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-709fbef1-b1ba-4aa9-a718-3aaf518b719d admin admin] The token cache is being invalidate because project f044bbb8bb2d4af9ae27d3b849e3a369 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:17:03.014381 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-709fbef1-b1ba-4aa9-a718-3aaf518b719d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:03.015012 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-709fbef1-b1ba-4aa9-a718-3aaf518b719d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:03.016790 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 963/1934] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:17:02 2022] DELETE /identity/v3/projects/f044bbb8bb2d4af9ae27d3b849e3a369 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:03.021793 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfeb267-9b0a-429f-b37b-886bbd1b55d1 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:03.022178 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfeb267-9b0a-429f-b37b-886bbd1b55d1 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:03.022343 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfeb267-9b0a-429f-b37b-886bbd1b55d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:03.041933 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-dbfeb267-9b0a-429f-b37b-886bbd1b55d1 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:03.043140 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-dbfeb267-9b0a-429f-b37b-886bbd1b55d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:03.080582 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:03.109627 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 972/1935] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:03.117823 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd5e0b-2d0f-4651-9918-1d0db1978772 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:03.118398 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd5e0b-2d0f-4651-9918-1d0db1978772 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:03.118491 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd5e0b-2d0f-4651-9918-1d0db1978772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:03.139486 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-efdd5e0b-2d0f-4651-9918-1d0db1978772 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:03.141256 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-efdd5e0b-2d0f-4651-9918-1d0db1978772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:03.150451 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 964/1936] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:03.155092 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfeb267-9b0a-429f-b37b-886bbd1b55d1 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:03.165760 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfeb267-9b0a-429f-b37b-886bbd1b55d1 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:03.171162 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d320a8-6c49-4108-8ad9-aac8f9a160ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:03.171834 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d320a8-6c49-4108-8ad9-aac8f9a160ca admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:03.172114 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d320a8-6c49-4108-8ad9-aac8f9a160ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:03.172387 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d320a8-6c49-4108-8ad9-aac8f9a160ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:03.173550 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d320a8-6c49-4108-8ad9-aac8f9a160ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:17:03.173880 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d320a8-6c49-4108-8ad9-aac8f9a160ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:03.175507 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d320a8-6c49-4108-8ad9-aac8f9a160ca admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:03.193915 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 973/1937] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:17:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:17:03.198823 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdd5e0b-2d0f-4651-9918-1d0db1978772 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:03.209070 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdd5e0b-2d0f-4651-9918-1d0db1978772 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:03.215360 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa094d5-95d8-4c33-aa4f-f07b17820e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:03.216054 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa094d5-95d8-4c33-aa4f-f07b17820e39 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:03.216337 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa094d5-95d8-4c33-aa4f-f07b17820e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:03.216608 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa094d5-95d8-4c33-aa4f-f07b17820e39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:03.217339 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa094d5-95d8-4c33-aa4f-f07b17820e39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:03.219267 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa094d5-95d8-4c33-aa4f-f07b17820e39 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:03.248992 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 965/1938] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:17:03 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:03.253495 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d320a8-6c49-4108-8ad9-aac8f9a160ca admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:03.263971 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d320a8-6c49-4108-8ad9-aac8f9a160ca admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:03.325172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c7bea-5cf0-49b1-b34e-1b9c2bd5ec12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:03.325861 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c7bea-5cf0-49b1-b34e-1b9c2bd5ec12 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:03.326145 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c7bea-5cf0-49b1-b34e-1b9c2bd5ec12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:03.326420 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c7bea-5cf0-49b1-b34e-1b9c2bd5ec12 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:03.327122 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c7bea-5cf0-49b1-b34e-1b9c2bd5ec12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:03.329122 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c7bea-5cf0-49b1-b34e-1b9c2bd5ec12 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:03.351581 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 974/1939] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:17:03 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:03.356197 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa094d5-95d8-4c33-aa4f-f07b17820e39 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:03.367399 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa094d5-95d8-4c33-aa4f-f07b17820e39 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:03.373438 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f9834f-e5fe-4c8b-a3ad-c9500353d0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:03.374114 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9834f-e5fe-4c8b-a3ad-c9500353d0ab admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:03.374548 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9834f-e5fe-4c8b-a3ad-c9500353d0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:03.374818 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9834f-e5fe-4c8b-a3ad-c9500353d0ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:03.375563 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f9834f-e5fe-4c8b-a3ad-c9500353d0ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:03.377214 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f9834f-e5fe-4c8b-a3ad-c9500353d0ab admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:03.388879 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 966/1940] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:03.393395 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c7bea-5cf0-49b1-b34e-1b9c2bd5ec12 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:03.404483 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c7bea-5cf0-49b1-b34e-1b9c2bd5ec12 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:03.410441 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca334ce1-dad3-4448-a786-b5009c95cddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:03.411144 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca334ce1-dad3-4448-a786-b5009c95cddb admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:03.411398 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca334ce1-dad3-4448-a786-b5009c95cddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:03.411669 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca334ce1-dad3-4448-a786-b5009c95cddb admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:03.412372 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca334ce1-dad3-4448-a786-b5009c95cddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:03.414010 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca334ce1-dad3-4448-a786-b5009c95cddb admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:03.427144 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 975/1941] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:03.432012 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f9834f-e5fe-4c8b-a3ad-c9500353d0ab admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:03.442742 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f9834f-e5fe-4c8b-a3ad-c9500353d0ab admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:03.447737 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ab349-42fa-4b44-bf02-3fd3c73c9b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:03.448639 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ab349-42fa-4b44-bf02-3fd3c73c9b1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:03.448938 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ab349-42fa-4b44-bf02-3fd3c73c9b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:03.449224 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ab349-42fa-4b44-bf02-3fd3c73c9b1c admin admin] PATH_INFO: `/v3/projects/f5a481ab707c47cca2b7ee7ee764a8da/users/6c85164fbc4b4113bc199ca986134da8/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:03.464961 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35ab349-42fa-4b44-bf02-3fd3c73c9b1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5a481ab707c47cca2b7ee7ee764a8da, user_id=6c85164fbc4b4113bc199ca986134da8, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:03.467548 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35ab349-42fa-4b44-bf02-3fd3c73c9b1c admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:03.481310 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 967/1942] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:17:03 2022] PUT /identity/v3/projects/f5a481ab707c47cca2b7ee7ee764a8da/users/6c85164fbc4b4113bc199ca986134da8/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:17:03.491817 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa018b11-e2d2-4276-9c1f-dcd04e037737 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:03.492104 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa018b11-e2d2-4276-9c1f-dcd04e037737 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:03.492381 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa018b11-e2d2-4276-9c1f-dcd04e037737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:03.521114 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-aa018b11-e2d2-4276-9c1f-dcd04e037737 None None] MFA Rules not processed for user `6c85164fbc4b4113bc199ca986134da8`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:03.522381 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-aa018b11-e2d2-4276-9c1f-dcd04e037737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:03.593646 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 976/1943] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:03.607040 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ab349-42fa-4b44-bf02-3fd3c73c9b1c admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:03.609146 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-a35ab349-42fa-4b44-bf02-3fd3c73c9b1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:03.651014 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:03.678774 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ab349-42fa-4b44-bf02-3fd3c73c9b1c admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:03.683802 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c09ffa-9ddb-4a29-9a68-3d2f117af234 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:03.684479 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c09ffa-9ddb-4a29-9a68-3d2f117af234 service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:03.684757 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c09ffa-9ddb-4a29-9a68-3d2f117af234 service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:03.685067 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c09ffa-9ddb-4a29-9a68-3d2f117af234 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:03.698392 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c09ffa-9ddb-4a29-9a68-3d2f117af234 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:03.699979 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c09ffa-9ddb-4a29-9a68-3d2f117af234 service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:03.710373 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 968/1944] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:03.732891 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa018b11-e2d2-4276-9c1f-dcd04e037737 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:03.736506 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c09ffa-9ddb-4a29-9a68-3d2f117af234 service cinder] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:03.738228 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-72c09ffa-9ddb-4a29-9a68-3d2f117af234 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:03.771379 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:03.811662 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa018b11-e2d2-4276-9c1f-dcd04e037737 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:03.815490 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c09ffa-9ddb-4a29-9a68-3d2f117af234 service cinder] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:03.816314 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877fb429-6a74-44f6-ae71-4b9b027f73cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:03.817242 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877fb429-6a74-44f6-ae71-4b9b027f73cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:03.817518 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877fb429-6a74-44f6-ae71-4b9b027f73cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:03.817789 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877fb429-6a74-44f6-ae71-4b9b027f73cd admin admin] PATH_INFO: `/v3/users/6c85164fbc4b4113bc199ca986134da8` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:03.818992 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877fb429-6a74-44f6-ae71-4b9b027f73cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c85164fbc4b4113bc199ca986134da8)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:03.820154 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fca4ec-2019-4d48-9ca9-193163306697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:03.820940 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877fb429-6a74-44f6-ae71-4b9b027f73cd admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:03.821147 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fca4ec-2019-4d48-9ca9-193163306697 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:03.821433 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fca4ec-2019-4d48-9ca9-193163306697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:03.821711 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fca4ec-2019-4d48-9ca9-193163306697 admin admin] PATH_INFO: `/v3/users/b0c77f6f328d47e9a7eb972cc4053818` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:03.822994 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fca4ec-2019-4d48-9ca9-193163306697 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0c77f6f328d47e9a7eb972cc4053818)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:03.824951 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fca4ec-2019-4d48-9ca9-193163306697 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:03.882787 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-877fb429-6a74-44f6-ae71-4b9b027f73cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c85164fbc4b4113bc199ca986134da8'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:03.888897 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-43fca4ec-2019-4d48-9ca9-193163306697 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0c77f6f328d47e9a7eb972cc4053818'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:03.891997 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-877fb429-6a74-44f6-ae71-4b9b027f73cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c85164fbc4b4113bc199ca986134da8'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:03.892798 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-877fb429-6a74-44f6-ae71-4b9b027f73cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c85164fbc4b4113bc199ca986134da8'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:03.893531 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-877fb429-6a74-44f6-ae71-4b9b027f73cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c85164fbc4b4113bc199ca986134da8'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:03.902283 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-43fca4ec-2019-4d48-9ca9-193163306697 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0c77f6f328d47e9a7eb972cc4053818'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:03.903029 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-877fb429-6a74-44f6-ae71-4b9b027f73cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c85164fbc4b4113bc199ca986134da8'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:03.903429 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-43fca4ec-2019-4d48-9ca9-193163306697 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0c77f6f328d47e9a7eb972cc4053818'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:03.904466 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-43fca4ec-2019-4d48-9ca9-193163306697 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0c77f6f328d47e9a7eb972cc4053818'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:03.916369 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-43fca4ec-2019-4d48-9ca9-193163306697 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0c77f6f328d47e9a7eb972cc4053818'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:03.922823 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 977/1945] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:17:03 2022] DELETE /identity/v3/users/6c85164fbc4b4113bc199ca986134da8 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:03.927206 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539bfa6-9f8d-40b8-87a8-c8ad8d139685 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:03.927500 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539bfa6-9f8d-40b8-87a8-c8ad8d139685 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:03.927786 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539bfa6-9f8d-40b8-87a8-c8ad8d139685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:03.943412 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 969/1946] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:17:03 2022] DELETE /identity/v3/users/b0c77f6f328d47e9a7eb972cc4053818 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:03.945834 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-d539bfa6-9f8d-40b8-87a8-c8ad8d139685 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:03.947116 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d539bfa6-9f8d-40b8-87a8-c8ad8d139685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:03.948518 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466a464-af06-485d-ad68-e9f1d068c495 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:03.948802 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466a464-af06-485d-ad68-e9f1d068c495 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:03.949163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5466a464-af06-485d-ad68-e9f1d068c495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:03.974860 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-5466a464-af06-485d-ad68-e9f1d068c495 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:03.976000 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-5466a464-af06-485d-ad68-e9f1d068c495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:04.023323 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 978/1947] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:17:04.035586 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539bfa6-9f8d-40b8-87a8-c8ad8d139685 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:04.037878 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d539bfa6-9f8d-40b8-87a8-c8ad8d139685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:04.055334 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:04.076605 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 970/1948] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:17:04.084982 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5466a464-af06-485d-ad68-e9f1d068c495 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:04.086845 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-5466a464-af06-485d-ad68-e9f1d068c495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:04.121121 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539bfa6-9f8d-40b8-87a8-c8ad8d139685 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:04.125554 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea705a1-88b2-4024-956f-ea998bab7fb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:04.126216 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea705a1-88b2-4024-956f-ea998bab7fb6 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:04.126690 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea705a1-88b2-4024-956f-ea998bab7fb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:04.126953 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea705a1-88b2-4024-956f-ea998bab7fb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:04.136715 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:04.139817 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea705a1-88b2-4024-956f-ea998bab7fb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:04.142150 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea705a1-88b2-4024-956f-ea998bab7fb6 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:04.163248 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 979/1949] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:17:04.163500 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5466a464-af06-485d-ad68-e9f1d068c495 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:04.163500 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86320610-e22d-41e9-9901-c893634d78e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:04.163500 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86320610-e22d-41e9-9901-c893634d78e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:04.163500 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86320610-e22d-41e9-9901-c893634d78e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:04.164212 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86320610-e22d-41e9-9901-c893634d78e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:04.173111 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86320610-e22d-41e9-9901-c893634d78e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:04.174817 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86320610-e22d-41e9-9901-c893634d78e1 service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:04.188467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 971/1950] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:17:04.567006 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea705a1-88b2-4024-956f-ea998bab7fb6 service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:04.568674 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-0ea705a1-88b2-4024-956f-ea998bab7fb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:04.595022 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea705a1-88b2-4024-956f-ea998bab7fb6 service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:04.605453 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e4ce0a-8d63-43a4-82ab-3177142d6f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:04.607153 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4ce0a-8d63-43a4-82ab-3177142d6f83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:04.607655 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4ce0a-8d63-43a4-82ab-3177142d6f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:04.611028 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4ce0a-8d63-43a4-82ab-3177142d6f83 admin admin] PATH_INFO: `/v3/projects/f5a481ab707c47cca2b7ee7ee764a8da` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:04.611996 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e4ce0a-8d63-43a4-82ab-3177142d6f83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5a481ab707c47cca2b7ee7ee764a8da)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:04.612500 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86320610-e22d-41e9-9901-c893634d78e1 service neutron] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:04.613963 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-86320610-e22d-41e9-9901-c893634d78e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:04.616182 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e4ce0a-8d63-43a4-82ab-3177142d6f83 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:04.645229 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86320610-e22d-41e9-9901-c893634d78e1 service neutron] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:04.662617 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6df468-3c22-4a13-b81c-f7ea17d1d5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:04.669758 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6df468-3c22-4a13-b81c-f7ea17d1d5ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:04.670104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6df468-3c22-4a13-b81c-f7ea17d1d5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:04.670599 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6df468-3c22-4a13-b81c-f7ea17d1d5ad admin admin] PATH_INFO: `/v3/projects/6bfa6be0235e4bbdbb610c1ab68059f7` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:04.673094 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6df468-3c22-4a13-b81c-f7ea17d1d5ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bfa6be0235e4bbdbb610c1ab68059f7)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:04.675744 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6df468-3c22-4a13-b81c-f7ea17d1d5ad admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:04.717760 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-02e4ce0a-8d63-43a4-82ab-3177142d6f83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5a481ab707c47cca2b7ee7ee764a8da', 'request_id': 'req-02e4ce0a-8d63-43a4-82ab-3177142d6f83'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:04.725437 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-02e4ce0a-8d63-43a4-82ab-3177142d6f83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5a481ab707c47cca2b7ee7ee764a8da', 'request_id': 'req-02e4ce0a-8d63-43a4-82ab-3177142d6f83'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:04.734050 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-02e4ce0a-8d63-43a4-82ab-3177142d6f83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5a481ab707c47cca2b7ee7ee764a8da', 'request_id': 'req-02e4ce0a-8d63-43a4-82ab-3177142d6f83'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:04.746862 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-bf6df468-3c22-4a13-b81c-f7ea17d1d5ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bfa6be0235e4bbdbb610c1ab68059f7', 'request_id': 'req-bf6df468-3c22-4a13-b81c-f7ea17d1d5ad'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:04.747975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-02e4ce0a-8d63-43a4-82ab-3177142d6f83 admin admin] The token cache is being invalidate because project f5a481ab707c47cca2b7ee7ee764a8da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:17:04.748373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-02e4ce0a-8d63-43a4-82ab-3177142d6f83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:04.749463 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-02e4ce0a-8d63-43a4-82ab-3177142d6f83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:04.752287 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 980/1951] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:17:04 2022] DELETE /identity/v3/projects/f5a481ab707c47cca2b7ee7ee764a8da => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:04.754207 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-bf6df468-3c22-4a13-b81c-f7ea17d1d5ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bfa6be0235e4bbdbb610c1ab68059f7', 'request_id': 'req-bf6df468-3c22-4a13-b81c-f7ea17d1d5ad'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:04.757713 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb030c-7004-40c7-8a25-67871d9f8428 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:04.758068 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb030c-7004-40c7-8a25-67871d9f8428 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:04.758462 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb030c-7004-40c7-8a25-67871d9f8428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:04.763455 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-bf6df468-3c22-4a13-b81c-f7ea17d1d5ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bfa6be0235e4bbdbb610c1ab68059f7', 'request_id': 'req-bf6df468-3c22-4a13-b81c-f7ea17d1d5ad'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:04.775209 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-bf6df468-3c22-4a13-b81c-f7ea17d1d5ad admin admin] The token cache is being invalidate because project 6bfa6be0235e4bbdbb610c1ab68059f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:17:04.775209 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-bf6df468-3c22-4a13-b81c-f7ea17d1d5ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:04.775977 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-bf6df468-3c22-4a13-b81c-f7ea17d1d5ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:04.777989 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 972/1952] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:17:04 2022] DELETE /identity/v3/projects/6bfa6be0235e4bbdbb610c1ab68059f7 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:17:04.779878 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-41fb030c-7004-40c7-8a25-67871d9f8428 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:04.781289 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-41fb030c-7004-40c7-8a25-67871d9f8428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:04.781885 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6df468-3c22-4a13-b81c-f7ea17d1d5ad admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:04.783211 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-bf6df468-3c22-4a13-b81c-f7ea17d1d5ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:04.847214 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 981/1953] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:04.856378 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a12791-b352-4999-ae46-c222d093744f None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:04.856665 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a12791-b352-4999-ae46-c222d093744f None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:04.857259 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a12791-b352-4999-ae46-c222d093744f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:04.857461 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6df468-3c22-4a13-b81c-f7ea17d1d5ad admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:04.863593 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a4774c-58c3-4208-a2ab-ecab3109b5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:04.864506 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a4774c-58c3-4208-a2ab-ecab3109b5c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:04.864963 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a4774c-58c3-4208-a2ab-ecab3109b5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:04.865278 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a4774c-58c3-4208-a2ab-ecab3109b5c1 admin admin] PATH_INFO: `/v3/users/59232c9123f14b898eec6a2d428b5897` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:04.866570 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a4774c-58c3-4208-a2ab-ecab3109b5c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59232c9123f14b898eec6a2d428b5897)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:04.868585 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a4774c-58c3-4208-a2ab-ecab3109b5c1 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:04.878410 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-c7a12791-b352-4999-ae46-c222d093744f None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:04.879618 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-c7a12791-b352-4999-ae46-c222d093744f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:04.888566 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 982/1954] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:17:04.893166 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a12791-b352-4999-ae46-c222d093744f None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:04.893486 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:04.904652 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a12791-b352-4999-ae46-c222d093744f None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:04.909941 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e2496-892e-48f0-8602-8715a01e30a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:04.910618 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e2496-892e-48f0-8602-8715a01e30a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:04.910916 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e2496-892e-48f0-8602-8715a01e30a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:04.911169 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e2496-892e-48f0-8602-8715a01e30a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:04.911957 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3e2496-892e-48f0-8602-8715a01e30a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:17:04.912205 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3e2496-892e-48f0-8602-8715a01e30a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:04.913900 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3e2496-892e-48f0-8602-8715a01e30a8 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:04.925219 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-81a4774c-58c3-4208-a2ab-ecab3109b5c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59232c9123f14b898eec6a2d428b5897'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:04.927594 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 983/1955] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:17:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:17:04.932270 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e2496-892e-48f0-8602-8715a01e30a8 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:04.934830 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-81a4774c-58c3-4208-a2ab-ecab3109b5c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59232c9123f14b898eec6a2d428b5897'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:04.935730 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-81a4774c-58c3-4208-a2ab-ecab3109b5c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59232c9123f14b898eec6a2d428b5897'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:04.936616 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-81a4774c-58c3-4208-a2ab-ecab3109b5c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59232c9123f14b898eec6a2d428b5897'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:04.943261 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e2496-892e-48f0-8602-8715a01e30a8 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:04.948395 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-81a4774c-58c3-4208-a2ab-ecab3109b5c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59232c9123f14b898eec6a2d428b5897'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:04.949007 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a109a177-3e09-4e38-8d7c-f284602d3373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:04.949714 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109a177-3e09-4e38-8d7c-f284602d3373 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:04.950005 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109a177-3e09-4e38-8d7c-f284602d3373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:04.950290 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109a177-3e09-4e38-8d7c-f284602d3373 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:04.950996 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a109a177-3e09-4e38-8d7c-f284602d3373 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:04.953170 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a109a177-3e09-4e38-8d7c-f284602d3373 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:04.968478 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 973/1956] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:17:04 2022] DELETE /identity/v3/users/59232c9123f14b898eec6a2d428b5897 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:04.986723 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 984/1957] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:17:04 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:17:04.991285 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a4774c-58c3-4208-a2ab-ecab3109b5c1 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:04.992575 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-81a4774c-58c3-4208-a2ab-ecab3109b5c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:05.077570 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a4774c-58c3-4208-a2ab-ecab3109b5c1 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:05.148276 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4ab5d9-c109-4a9c-86b2-d2d4f0ce1564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:05.149446 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ab5d9-c109-4a9c-86b2-d2d4f0ce1564 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:05.149738 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ab5d9-c109-4a9c-86b2-d2d4f0ce1564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:05.150043 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ab5d9-c109-4a9c-86b2-d2d4f0ce1564 admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:05.151121 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4ab5d9-c109-4a9c-86b2-d2d4f0ce1564 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:05.153347 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4ab5d9-c109-4a9c-86b2-d2d4f0ce1564 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:05.180622 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 974/1958] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:17:04 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:05.184918 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a109a177-3e09-4e38-8d7c-f284602d3373 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:05.215370 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a109a177-3e09-4e38-8d7c-f284602d3373 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:05.224654 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88645efd-eb24-4dfc-9acc-7997ba581578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:05.225718 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88645efd-eb24-4dfc-9acc-7997ba581578 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:05.225994 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88645efd-eb24-4dfc-9acc-7997ba581578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:05.226268 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88645efd-eb24-4dfc-9acc-7997ba581578 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:05.227040 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88645efd-eb24-4dfc-9acc-7997ba581578 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:05.229588 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88645efd-eb24-4dfc-9acc-7997ba581578 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:05.243360 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 985/1959] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:05.249433 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4ab5d9-c109-4a9c-86b2-d2d4f0ce1564 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:05.264415 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4ab5d9-c109-4a9c-86b2-d2d4f0ce1564 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:05.272266 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd8cf0-dfc9-45ff-91c6-4cb0577a3918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:05.273290 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd8cf0-dfc9-45ff-91c6-4cb0577a3918 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:05.273739 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd8cf0-dfc9-45ff-91c6-4cb0577a3918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:05.274828 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd8cf0-dfc9-45ff-91c6-4cb0577a3918 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:05.276369 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fd8cf0-dfc9-45ff-91c6-4cb0577a3918 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:05.280818 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fd8cf0-dfc9-45ff-91c6-4cb0577a3918 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:05.305004 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 975/1960] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:05.306812 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88645efd-eb24-4dfc-9acc-7997ba581578 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:05.325025 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88645efd-eb24-4dfc-9acc-7997ba581578 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:05.329681 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd8cf0-dfc9-45ff-91c6-4cb0577a3918 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:05.331938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca038119-4cd3-427f-aa97-fb5259592f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:05.333171 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca038119-4cd3-427f-aa97-fb5259592f82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:05.334781 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca038119-4cd3-427f-aa97-fb5259592f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:05.334781 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca038119-4cd3-427f-aa97-fb5259592f82 admin admin] PATH_INFO: `/v3/projects/6212e66c6a8642879fcda0a07f4aa68a/users/6be5074fcecb45918bc904d9baa63aa5/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:05.338854 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-c6fd8cf0-dfc9-45ff-91c6-4cb0577a3918 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:05.363072 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd8cf0-dfc9-45ff-91c6-4cb0577a3918 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:05.371755 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca038119-4cd3-427f-aa97-fb5259592f82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6212e66c6a8642879fcda0a07f4aa68a, user_id=6be5074fcecb45918bc904d9baa63aa5, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:05.373720 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6168da6-1120-4605-8c3f-ac1af02299d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:05.375871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6168da6-1120-4605-8c3f-ac1af02299d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:05.376336 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6168da6-1120-4605-8c3f-ac1af02299d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:05.376798 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6168da6-1120-4605-8c3f-ac1af02299d1 admin admin] PATH_INFO: `/v3/projects/56df7ea73bf64ae3ab7d91a0c6c475d2` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:05.377420 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca038119-4cd3-427f-aa97-fb5259592f82 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:05.379149 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6168da6-1120-4605-8c3f-ac1af02299d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56df7ea73bf64ae3ab7d91a0c6c475d2)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:05.382798 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6168da6-1120-4605-8c3f-ac1af02299d1 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:05.399150 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 986/1961] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:17:05 2022] PUT /identity/v3/projects/6212e66c6a8642879fcda0a07f4aa68a/users/6be5074fcecb45918bc904d9baa63aa5/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:05.411254 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0360d2-c446-452e-afca-c91679cc5b22 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:05.411594 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0360d2-c446-452e-afca-c91679cc5b22 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:05.411874 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0360d2-c446-452e-afca-c91679cc5b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:05.419614 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:05.422765 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:05.446692 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-4c0360d2-c446-452e-afca-c91679cc5b22 None None] MFA Rules not processed for user `6be5074fcecb45918bc904d9baa63aa5`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:05.447961 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-4c0360d2-c446-452e-afca-c91679cc5b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:05.463771 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-d6168da6-1120-4605-8c3f-ac1af02299d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56df7ea73bf64ae3ab7d91a0c6c475d2', 'request_id': 'req-d6168da6-1120-4605-8c3f-ac1af02299d1'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:05.471334 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-d6168da6-1120-4605-8c3f-ac1af02299d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56df7ea73bf64ae3ab7d91a0c6c475d2', 'request_id': 'req-d6168da6-1120-4605-8c3f-ac1af02299d1'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:05.482034 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-d6168da6-1120-4605-8c3f-ac1af02299d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56df7ea73bf64ae3ab7d91a0c6c475d2', 'request_id': 'req-d6168da6-1120-4605-8c3f-ac1af02299d1'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:05.495155 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-d6168da6-1120-4605-8c3f-ac1af02299d1 admin admin] The token cache is being invalidate because project 56df7ea73bf64ae3ab7d91a0c6c475d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:17:05.495564 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-d6168da6-1120-4605-8c3f-ac1af02299d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:05.496526 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-d6168da6-1120-4605-8c3f-ac1af02299d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:05.499741 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 976/1962] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:17:05 2022] DELETE /identity/v3/projects/56df7ea73bf64ae3ab7d91a0c6c475d2 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:17:05.504233 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6168da6-1120-4605-8c3f-ac1af02299d1 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:05.506856 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-d6168da6-1120-4605-8c3f-ac1af02299d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:05.531345 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 987/1963] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:17:05.546413 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0360d2-c446-452e-afca-c91679cc5b22 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:05.547886 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-4c0360d2-c446-452e-afca-c91679cc5b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:05.584860 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6168da6-1120-4605-8c3f-ac1af02299d1 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:05.590679 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd44ccc4-e7b6-471f-8a7d-6e9195d37ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:05.591576 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd44ccc4-e7b6-471f-8a7d-6e9195d37ac2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:05.592009 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd44ccc4-e7b6-471f-8a7d-6e9195d37ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:05.592291 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd44ccc4-e7b6-471f-8a7d-6e9195d37ac2 admin admin] PATH_INFO: `/v3/users/63c66af4436a482182ed082aacf60fb1` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:05.593592 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd44ccc4-e7b6-471f-8a7d-6e9195d37ac2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63c66af4436a482182ed082aacf60fb1)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:05.595557 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd44ccc4-e7b6-471f-8a7d-6e9195d37ac2 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:05.617695 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0360d2-c446-452e-afca-c91679cc5b22 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:05.621991 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5832481-5640-4822-9b5b-c69da3a57e41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:05.622829 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5832481-5640-4822-9b5b-c69da3a57e41 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:05.623214 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5832481-5640-4822-9b5b-c69da3a57e41 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:05.623397 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5832481-5640-4822-9b5b-c69da3a57e41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:05.634131 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5832481-5640-4822-9b5b-c69da3a57e41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:05.636186 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5832481-5640-4822-9b5b-c69da3a57e41 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:05.646489 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 988/1964] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:05 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:05.649625 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-fd44ccc4-e7b6-471f-8a7d-6e9195d37ac2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63c66af4436a482182ed082aacf60fb1'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:05.657779 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-fd44ccc4-e7b6-471f-8a7d-6e9195d37ac2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63c66af4436a482182ed082aacf60fb1'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:05.658523 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-fd44ccc4-e7b6-471f-8a7d-6e9195d37ac2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63c66af4436a482182ed082aacf60fb1'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:05.659135 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-fd44ccc4-e7b6-471f-8a7d-6e9195d37ac2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63c66af4436a482182ed082aacf60fb1'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:05.668089 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-fd44ccc4-e7b6-471f-8a7d-6e9195d37ac2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63c66af4436a482182ed082aacf60fb1'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:05.686531 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 977/1965] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:17:05 2022] DELETE /identity/v3/users/63c66af4436a482182ed082aacf60fb1 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:17:06.001318 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5832481-5640-4822-9b5b-c69da3a57e41 service cinder] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:06.027840 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5832481-5640-4822-9b5b-c69da3a57e41 service cinder] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:06.091407 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44eb7f5-d561-456a-b13c-ae9617bf1604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:06.092294 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44eb7f5-d561-456a-b13c-ae9617bf1604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:06.092573 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44eb7f5-d561-456a-b13c-ae9617bf1604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:06.092847 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44eb7f5-d561-456a-b13c-ae9617bf1604 admin admin] PATH_INFO: `/v3/projects/b129240935544b9487c2e2fdb2414353` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:06.094196 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44eb7f5-d561-456a-b13c-ae9617bf1604 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b129240935544b9487c2e2fdb2414353)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:06.096136 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44eb7f5-d561-456a-b13c-ae9617bf1604 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:06.138309 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:06.151008 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-e44eb7f5-d561-456a-b13c-ae9617bf1604 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b129240935544b9487c2e2fdb2414353', 'request_id': 'req-e44eb7f5-d561-456a-b13c-ae9617bf1604'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:06.158546 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-e44eb7f5-d561-456a-b13c-ae9617bf1604 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b129240935544b9487c2e2fdb2414353', 'request_id': 'req-e44eb7f5-d561-456a-b13c-ae9617bf1604'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:06.167193 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-e44eb7f5-d561-456a-b13c-ae9617bf1604 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b129240935544b9487c2e2fdb2414353', 'request_id': 'req-e44eb7f5-d561-456a-b13c-ae9617bf1604'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:06.178256 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-e44eb7f5-d561-456a-b13c-ae9617bf1604 admin admin] The token cache is being invalidate because project b129240935544b9487c2e2fdb2414353 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:17:06.178546 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-e44eb7f5-d561-456a-b13c-ae9617bf1604 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:06.179320 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-e44eb7f5-d561-456a-b13c-ae9617bf1604 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:06.184277 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 989/1966] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:17:05 2022] DELETE /identity/v3/projects/b129240935544b9487c2e2fdb2414353 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:06.186010 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d909c6-7695-4254-a969-51dee8ba2c4e None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:06.186682 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d909c6-7695-4254-a969-51dee8ba2c4e None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:06.186682 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d909c6-7695-4254-a969-51dee8ba2c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:06.206329 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-17d909c6-7695-4254-a969-51dee8ba2c4e None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:06.207525 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-17d909c6-7695-4254-a969-51dee8ba2c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:06.231405 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:06.275153 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 978/1967] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:17:06.283258 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd4ca2-c4d6-4d20-a75d-ec491e706fda None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:06.283754 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd4ca2-c4d6-4d20-a75d-ec491e706fda None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:06.283754 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd4ca2-c4d6-4d20-a75d-ec491e706fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:06.303454 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-50dd4ca2-c4d6-4d20-a75d-ec491e706fda None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:06.304700 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-50dd4ca2-c4d6-4d20-a75d-ec491e706fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:06.313839 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 990/1968] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:06.318309 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d909c6-7695-4254-a969-51dee8ba2c4e None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:06.330245 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d909c6-7695-4254-a969-51dee8ba2c4e None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:06.336012 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef18e70-43d4-4cd0-92d7-60febe653e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:06.336893 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef18e70-43d4-4cd0-92d7-60febe653e6a admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:06.338168 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef18e70-43d4-4cd0-92d7-60febe653e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:06.338449 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef18e70-43d4-4cd0-92d7-60febe653e6a admin admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:06.339213 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef18e70-43d4-4cd0-92d7-60febe653e6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:17:06.339517 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef18e70-43d4-4cd0-92d7-60febe653e6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:06.341270 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef18e70-43d4-4cd0-92d7-60febe653e6a admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:06.356859 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 979/1969] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:17:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:17:06.361356 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd4ca2-c4d6-4d20-a75d-ec491e706fda None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:06.371843 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd4ca2-c4d6-4d20-a75d-ec491e706fda None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:06.377047 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70770186-06dd-4e68-b6d1-be36e4514936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:06.377719 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70770186-06dd-4e68-b6d1-be36e4514936 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:06.378002 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70770186-06dd-4e68-b6d1-be36e4514936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:06.378275 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70770186-06dd-4e68-b6d1-be36e4514936 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:06.378981 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70770186-06dd-4e68-b6d1-be36e4514936 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:06.380917 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70770186-06dd-4e68-b6d1-be36e4514936 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:06.418242 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 991/1970] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:17:06 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:06.422375 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef18e70-43d4-4cd0-92d7-60febe653e6a admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:06.432917 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef18e70-43d4-4cd0-92d7-60febe653e6a admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:06.498509 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd1fc80-6cd7-4f07-a444-9a209358ae0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:06.499203 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd1fc80-6cd7-4f07-a444-9a209358ae0a admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:06.499490 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd1fc80-6cd7-4f07-a444-9a209358ae0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:06.499771 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd1fc80-6cd7-4f07-a444-9a209358ae0a admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:06.500473 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd1fc80-6cd7-4f07-a444-9a209358ae0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:06.502465 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd1fc80-6cd7-4f07-a444-9a209358ae0a admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:06.529870 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 980/1971] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:17:06 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:06.531160 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70770186-06dd-4e68-b6d1-be36e4514936 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:06.543085 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70770186-06dd-4e68-b6d1-be36e4514936 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:06.548757 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656562c4-8c63-40d9-a598-468f28e1ed6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:06.549491 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656562c4-8c63-40d9-a598-468f28e1ed6f admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:06.549778 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656562c4-8c63-40d9-a598-468f28e1ed6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:06.550052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656562c4-8c63-40d9-a598-468f28e1ed6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:06.550774 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656562c4-8c63-40d9-a598-468f28e1ed6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:06.552508 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656562c4-8c63-40d9-a598-468f28e1ed6f admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:06.565052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 992/1972] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:06.569147 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd1fc80-6cd7-4f07-a444-9a209358ae0a admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:06.579586 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd1fc80-6cd7-4f07-a444-9a209358ae0a admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:06.584278 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506fda95-6181-4a35-b9ba-c161b181396b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:06.584987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506fda95-6181-4a35-b9ba-c161b181396b admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:06.585276 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506fda95-6181-4a35-b9ba-c161b181396b admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:06.585551 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506fda95-6181-4a35-b9ba-c161b181396b admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:06.586292 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506fda95-6181-4a35-b9ba-c161b181396b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:06.588505 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506fda95-6181-4a35-b9ba-c161b181396b admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:06.599044 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 981/1973] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:17:06.603266 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656562c4-8c63-40d9-a598-468f28e1ed6f admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:06.613562 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656562c4-8c63-40d9-a598-468f28e1ed6f admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:06.618334 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0037c725-4804-4140-ba96-b11cbd4d9b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:06.619606 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037c725-4804-4140-ba96-b11cbd4d9b1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:06.619894 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037c725-4804-4140-ba96-b11cbd4d9b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:06.620177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037c725-4804-4140-ba96-b11cbd4d9b1e admin admin] PATH_INFO: `/v3/projects/6b53f279afe9432a87aed0df2b4272a4/users/c9eb3e21e293403993d9093ed85926f4/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:06.636150 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0037c725-4804-4140-ba96-b11cbd4d9b1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b53f279afe9432a87aed0df2b4272a4, user_id=c9eb3e21e293403993d9093ed85926f4, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:06.638879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0037c725-4804-4140-ba96-b11cbd4d9b1e admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:06.655256 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 993/1974] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:17:06 2022] PUT /identity/v3/projects/6b53f279afe9432a87aed0df2b4272a4/users/c9eb3e21e293403993d9093ed85926f4/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:06.666449 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18f9d6-9f5b-41b6-af19-7a461aed2fa8 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:06.666737 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18f9d6-9f5b-41b6-af19-7a461aed2fa8 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:06.667016 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18f9d6-9f5b-41b6-af19-7a461aed2fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:06.694795 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-bd18f9d6-9f5b-41b6-af19-7a461aed2fa8 None None] MFA Rules not processed for user `c9eb3e21e293403993d9093ed85926f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:06.696044 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-bd18f9d6-9f5b-41b6-af19-7a461aed2fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:06.774503 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 982/1975] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:06.784231 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0037c725-4804-4140-ba96-b11cbd4d9b1e admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:06.786320 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-0037c725-4804-4140-ba96-b11cbd4d9b1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:06.855477 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0037c725-4804-4140-ba96-b11cbd4d9b1e admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:06.860392 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e101e85-9ffc-447e-a6f8-09870e9d854e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:06.861191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e101e85-9ffc-447e-a6f8-09870e9d854e service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:06.861522 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e101e85-9ffc-447e-a6f8-09870e9d854e service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:06.861843 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e101e85-9ffc-447e-a6f8-09870e9d854e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:06.875467 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e101e85-9ffc-447e-a6f8-09870e9d854e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:06.878194 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e101e85-9ffc-447e-a6f8-09870e9d854e service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:06.894784 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 994/1976] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:16.767238 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=87,keystone:INSERT=3 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:16.888809 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=64,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:17.985547 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd18f9d6-9f5b-41b6-af19-7a461aed2fa8 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:17.987315 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-bd18f9d6-9f5b-41b6-af19-7a461aed2fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:18.077737 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd18f9d6-9f5b-41b6-af19-7a461aed2fa8 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:18.083599 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d791ab00-319d-44fb-aa5c-1b3552ab48b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:18.084527 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791ab00-319d-44fb-aa5c-1b3552ab48b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:18.084815 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791ab00-319d-44fb-aa5c-1b3552ab48b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:18.085131 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791ab00-319d-44fb-aa5c-1b3552ab48b4 admin admin] PATH_INFO: `/v3/users/4482c4806e724f899c890e10d0223f0d` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:18.086725 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d791ab00-319d-44fb-aa5c-1b3552ab48b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4482c4806e724f899c890e10d0223f0d)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:18.090030 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d791ab00-319d-44fb-aa5c-1b3552ab48b4 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:18.146433 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-d791ab00-319d-44fb-aa5c-1b3552ab48b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4482c4806e724f899c890e10d0223f0d'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:18.153742 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-d791ab00-319d-44fb-aa5c-1b3552ab48b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4482c4806e724f899c890e10d0223f0d'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:18.154365 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-d791ab00-319d-44fb-aa5c-1b3552ab48b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4482c4806e724f899c890e10d0223f0d'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:18.154965 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-d791ab00-319d-44fb-aa5c-1b3552ab48b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4482c4806e724f899c890e10d0223f0d'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:18.163224 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-d791ab00-319d-44fb-aa5c-1b3552ab48b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4482c4806e724f899c890e10d0223f0d'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:18.183856 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 983/1977] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:17:17 2022] DELETE /identity/v3/users/4482c4806e724f899c890e10d0223f0d => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:18.188572 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7874945-a6de-4ff1-b47b-782a3d3e65c1 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:18.189091 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7874945-a6de-4ff1-b47b-782a3d3e65c1 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:18.189303 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7874945-a6de-4ff1-b47b-782a3d3e65c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:18.207439 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-c7874945-a6de-4ff1-b47b-782a3d3e65c1 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:18.208674 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-c7874945-a6de-4ff1-b47b-782a3d3e65c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:18.279410 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 995/1978] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:18.291276 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d791ab00-319d-44fb-aa5c-1b3552ab48b4 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:18.293207 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-d791ab00-319d-44fb-aa5c-1b3552ab48b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:18.380023 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d791ab00-319d-44fb-aa5c-1b3552ab48b4 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:18.384786 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade2c6c7-5e56-4571-a7ae-fe1d1077b32d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:18.387057 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2c6c7-5e56-4571-a7ae-fe1d1077b32d service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:18.387452 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2c6c7-5e56-4571-a7ae-fe1d1077b32d service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:18.387807 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2c6c7-5e56-4571-a7ae-fe1d1077b32d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:18.398798 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade2c6c7-5e56-4571-a7ae-fe1d1077b32d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:18.401011 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade2c6c7-5e56-4571-a7ae-fe1d1077b32d service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:18.412784 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 984/1979] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:18.724550 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7874945-a6de-4ff1-b47b-782a3d3e65c1 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:18.726542 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-c7874945-a6de-4ff1-b47b-782a3d3e65c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:18.743102 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7874945-a6de-4ff1-b47b-782a3d3e65c1 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:18.748648 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9462e6-d3bb-4379-a7ad-657ae99c5b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:18.749785 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9462e6-d3bb-4379-a7ad-657ae99c5b4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:18.750208 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9462e6-d3bb-4379-a7ad-657ae99c5b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:18.750484 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9462e6-d3bb-4379-a7ad-657ae99c5b4d admin admin] PATH_INFO: `/v3/projects/201d4043c8d9474ebc87b0fe763cc1f9` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:18.751745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9462e6-d3bb-4379-a7ad-657ae99c5b4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=201d4043c8d9474ebc87b0fe763cc1f9)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:18.753956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9462e6-d3bb-4379-a7ad-657ae99c5b4d admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:18.817358 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ae9462e6-d3bb-4379-a7ad-657ae99c5b4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '201d4043c8d9474ebc87b0fe763cc1f9', 'request_id': 'req-ae9462e6-d3bb-4379-a7ad-657ae99c5b4d'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:18.825240 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ae9462e6-d3bb-4379-a7ad-657ae99c5b4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '201d4043c8d9474ebc87b0fe763cc1f9', 'request_id': 'req-ae9462e6-d3bb-4379-a7ad-657ae99c5b4d'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:18.834390 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ae9462e6-d3bb-4379-a7ad-657ae99c5b4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '201d4043c8d9474ebc87b0fe763cc1f9', 'request_id': 'req-ae9462e6-d3bb-4379-a7ad-657ae99c5b4d'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:18.846651 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ae9462e6-d3bb-4379-a7ad-657ae99c5b4d admin admin] The token cache is being invalidate because project 201d4043c8d9474ebc87b0fe763cc1f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:17:18.846942 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ae9462e6-d3bb-4379-a7ad-657ae99c5b4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:18.847601 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ae9462e6-d3bb-4379-a7ad-657ae99c5b4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:18.849283 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 996/1980] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:17:18 2022] DELETE /identity/v3/projects/201d4043c8d9474ebc87b0fe763cc1f9 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:18.854694 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af5f72-7da0-41f5-bec2-82d250ad1974 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:18.854986 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af5f72-7da0-41f5-bec2-82d250ad1974 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:18.855266 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af5f72-7da0-41f5-bec2-82d250ad1974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:18.876065 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-e1af5f72-7da0-41f5-bec2-82d250ad1974 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:18.877276 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-e1af5f72-7da0-41f5-bec2-82d250ad1974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:18.900387 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:18.959546 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 985/1981] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:18.967836 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25311751-2258-4a19-a667-e8757ca2905f None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:18.968350 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25311751-2258-4a19-a667-e8757ca2905f None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:18.968776 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25311751-2258-4a19-a667-e8757ca2905f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:19.016045 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-25311751-2258-4a19-a667-e8757ca2905f None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:19.017414 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-25311751-2258-4a19-a667-e8757ca2905f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:19.033704 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 997/1982] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:17:19.038113 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af5f72-7da0-41f5-bec2-82d250ad1974 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:19.052497 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af5f72-7da0-41f5-bec2-82d250ad1974 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:19.065004 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5448aaa-8e08-4793-a35f-4c45b594031a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:19.066397 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5448aaa-8e08-4793-a35f-4c45b594031a admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:19.067161 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5448aaa-8e08-4793-a35f-4c45b594031a admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:19.067719 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5448aaa-8e08-4793-a35f-4c45b594031a admin admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:19.069179 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5448aaa-8e08-4793-a35f-4c45b594031a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:17:19.069784 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5448aaa-8e08-4793-a35f-4c45b594031a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:19.072416 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5448aaa-8e08-4793-a35f-4c45b594031a admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:19.091008 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 986/1983] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:17:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:17:19.095896 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25311751-2258-4a19-a667-e8757ca2905f None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:19.106160 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25311751-2258-4a19-a667-e8757ca2905f None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:19.111502 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0b22d-325a-4809-bb3a-7ce154eb6d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:19.112902 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0b22d-325a-4809-bb3a-7ce154eb6d54 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:19.113217 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0b22d-325a-4809-bb3a-7ce154eb6d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:19.113488 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0b22d-325a-4809-bb3a-7ce154eb6d54 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:19.114199 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e0b22d-325a-4809-bb3a-7ce154eb6d54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:19.116353 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e0b22d-325a-4809-bb3a-7ce154eb6d54 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:19.153238 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 998/1984] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:17:19 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:19.155503 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5448aaa-8e08-4793-a35f-4c45b594031a admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:19.168561 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5448aaa-8e08-4793-a35f-4c45b594031a admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:19.244342 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c081c0-cad0-4116-9b79-53b6db47d41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:19.245467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c081c0-cad0-4116-9b79-53b6db47d41e admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:19.245957 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c081c0-cad0-4116-9b79-53b6db47d41e admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:19.246843 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c081c0-cad0-4116-9b79-53b6db47d41e admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:19.247796 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c081c0-cad0-4116-9b79-53b6db47d41e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:19.250239 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c081c0-cad0-4116-9b79-53b6db47d41e admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:19.278446 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 987/1985] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:17:19 2022] POST /identity/v3/users => generated 293 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:19.279730 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0b22d-325a-4809-bb3a-7ce154eb6d54 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:19.294175 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0b22d-325a-4809-bb3a-7ce154eb6d54 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:19.299057 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78963e2-cbd5-4361-81ff-c4a65eba849e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:19.299732 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78963e2-cbd5-4361-81ff-c4a65eba849e admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:19.300015 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78963e2-cbd5-4361-81ff-c4a65eba849e admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:19.300294 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78963e2-cbd5-4361-81ff-c4a65eba849e admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:19.301070 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78963e2-cbd5-4361-81ff-c4a65eba849e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:19.302683 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78963e2-cbd5-4361-81ff-c4a65eba849e admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:19.318296 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 999/1986] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:17:19.322901 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c081c0-cad0-4116-9b79-53b6db47d41e admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:19.333228 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c081c0-cad0-4116-9b79-53b6db47d41e admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:19.339136 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a0569-2a83-43f0-8f13-03dd7b916203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:19.339888 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a0569-2a83-43f0-8f13-03dd7b916203 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:19.340176 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a0569-2a83-43f0-8f13-03dd7b916203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:19.340580 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a0569-2a83-43f0-8f13-03dd7b916203 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:19.341363 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a0569-2a83-43f0-8f13-03dd7b916203 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:19.343005 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a0569-2a83-43f0-8f13-03dd7b916203 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:19.362823 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 988/1987] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:19 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:19.366781 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78963e2-cbd5-4361-81ff-c4a65eba849e admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:19.378581 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78963e2-cbd5-4361-81ff-c4a65eba849e admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:19.384388 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86369bb0-4e68-4fab-8446-55a4d115a187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:19.385382 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86369bb0-4e68-4fab-8446-55a4d115a187 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:19.385666 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86369bb0-4e68-4fab-8446-55a4d115a187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:19.385953 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86369bb0-4e68-4fab-8446-55a4d115a187 admin admin] PATH_INFO: `/v3/projects/4833ce998ff047cfb90a2fc82abf277e/users/10262ff17c7f476a8a4ae14048c82c18/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:19.405840 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86369bb0-4e68-4fab-8446-55a4d115a187 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4833ce998ff047cfb90a2fc82abf277e, user_id=10262ff17c7f476a8a4ae14048c82c18, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:19.405840 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86369bb0-4e68-4fab-8446-55a4d115a187 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:19.421345 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1000/1988] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:17:19 2022] PUT /identity/v3/projects/4833ce998ff047cfb90a2fc82abf277e/users/10262ff17c7f476a8a4ae14048c82c18/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:19.432441 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400fd87d-b5ab-4478-9718-43ec33e3c304 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:19.432718 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400fd87d-b5ab-4478-9718-43ec33e3c304 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:19.433027 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400fd87d-b5ab-4478-9718-43ec33e3c304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:19.491605 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-400fd87d-b5ab-4478-9718-43ec33e3c304 None None] MFA Rules not processed for user `10262ff17c7f476a8a4ae14048c82c18`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:19.492907 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-400fd87d-b5ab-4478-9718-43ec33e3c304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:19.597085 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 989/1989] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:17:19.630645 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86369bb0-4e68-4fab-8446-55a4d115a187 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:19.632486 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-86369bb0-4e68-4fab-8446-55a4d115a187 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:19.659429 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:19.705352 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86369bb0-4e68-4fab-8446-55a4d115a187 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:19.709358 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209ea54-3264-464c-9b65-c45915690d64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:19.710017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209ea54-3264-464c-9b65-c45915690d64 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:19.710371 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209ea54-3264-464c-9b65-c45915690d64 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:19.710541 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1209ea54-3264-464c-9b65-c45915690d64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:19.720488 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1209ea54-3264-464c-9b65-c45915690d64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:19.721964 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1209ea54-3264-464c-9b65-c45915690d64 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:19.733769 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1001/1990] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:24.592766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400fd87d-b5ab-4478-9718-43ec33e3c304 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:24.594716 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-400fd87d-b5ab-4478-9718-43ec33e3c304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:24.615648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:24.667718 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400fd87d-b5ab-4478-9718-43ec33e3c304 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:24.673363 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067f2f4-4b15-4f24-b5f1-547c01bb8d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:24.674194 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067f2f4-4b15-4f24-b5f1-547c01bb8d98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:24.674476 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067f2f4-4b15-4f24-b5f1-547c01bb8d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:24.674748 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067f2f4-4b15-4f24-b5f1-547c01bb8d98 admin admin] PATH_INFO: `/v3/users/db0e1b60083640bd89b9b4287fdb9d28` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:24.676109 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9067f2f4-4b15-4f24-b5f1-547c01bb8d98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db0e1b60083640bd89b9b4287fdb9d28)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:24.678181 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9067f2f4-4b15-4f24-b5f1-547c01bb8d98 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:24.735497 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-9067f2f4-4b15-4f24-b5f1-547c01bb8d98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db0e1b60083640bd89b9b4287fdb9d28'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:24.742534 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-9067f2f4-4b15-4f24-b5f1-547c01bb8d98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db0e1b60083640bd89b9b4287fdb9d28'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:24.743426 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-9067f2f4-4b15-4f24-b5f1-547c01bb8d98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db0e1b60083640bd89b9b4287fdb9d28'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:24.744172 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-9067f2f4-4b15-4f24-b5f1-547c01bb8d98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db0e1b60083640bd89b9b4287fdb9d28'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:24.753907 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-9067f2f4-4b15-4f24-b5f1-547c01bb8d98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db0e1b60083640bd89b9b4287fdb9d28'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:24.775619 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 990/1991] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:17:24 2022] DELETE /identity/v3/users/db0e1b60083640bd89b9b4287fdb9d28 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:24.965030 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209ea54-3264-464c-9b65-c45915690d64 service cinder] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:24.987408 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1209ea54-3264-464c-9b65-c45915690d64 service cinder] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:25.066523 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6088f42-5b87-408a-bf85-298ac9208c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:25.067411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6088f42-5b87-408a-bf85-298ac9208c99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:25.067708 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6088f42-5b87-408a-bf85-298ac9208c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:25.067968 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6088f42-5b87-408a-bf85-298ac9208c99 admin admin] PATH_INFO: `/v3/projects/a2e58fc8b6e14185b4409884a5701316` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:25.069182 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6088f42-5b87-408a-bf85-298ac9208c99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2e58fc8b6e14185b4409884a5701316)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:25.071168 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6088f42-5b87-408a-bf85-298ac9208c99 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:25.124212 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-e6088f42-5b87-408a-bf85-298ac9208c99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2e58fc8b6e14185b4409884a5701316', 'request_id': 'req-e6088f42-5b87-408a-bf85-298ac9208c99'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:25.130982 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-e6088f42-5b87-408a-bf85-298ac9208c99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2e58fc8b6e14185b4409884a5701316', 'request_id': 'req-e6088f42-5b87-408a-bf85-298ac9208c99'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:25.138913 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-e6088f42-5b87-408a-bf85-298ac9208c99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2e58fc8b6e14185b4409884a5701316', 'request_id': 'req-e6088f42-5b87-408a-bf85-298ac9208c99'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:25.150828 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-e6088f42-5b87-408a-bf85-298ac9208c99 admin admin] The token cache is being invalidate because project a2e58fc8b6e14185b4409884a5701316 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:17:25.151115 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-e6088f42-5b87-408a-bf85-298ac9208c99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:25.151906 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-e6088f42-5b87-408a-bf85-298ac9208c99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:25.153507 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1002/1992] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:17:24 2022] DELETE /identity/v3/projects/a2e58fc8b6e14185b4409884a5701316 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:25.158843 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36568338-a98a-4b66-8153-fcae0868b503 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:25.159118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36568338-a98a-4b66-8153-fcae0868b503 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:25.159425 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36568338-a98a-4b66-8153-fcae0868b503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:25.179513 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-36568338-a98a-4b66-8153-fcae0868b503 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:25.180844 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-36568338-a98a-4b66-8153-fcae0868b503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:25.252582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 991/1993] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:25.261015 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9db793-46f1-40a8-9a5d-4486040f5bb6 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:25.261306 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9db793-46f1-40a8-9a5d-4486040f5bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:25.261578 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9db793-46f1-40a8-9a5d-4486040f5bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:25.291512 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-1b9db793-46f1-40a8-9a5d-4486040f5bb6 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:25.292865 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-1b9db793-46f1-40a8-9a5d-4486040f5bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:25.303171 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1003/1994] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:17:25.308192 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36568338-a98a-4b66-8153-fcae0868b503 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:25.320842 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36568338-a98a-4b66-8153-fcae0868b503 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:25.327978 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ebf9da-630c-46a2-8b7f-10b49e5cb846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:25.329702 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ebf9da-630c-46a2-8b7f-10b49e5cb846 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:25.330205 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ebf9da-630c-46a2-8b7f-10b49e5cb846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:25.330517 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ebf9da-630c-46a2-8b7f-10b49e5cb846 admin admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:25.331401 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ebf9da-630c-46a2-8b7f-10b49e5cb846 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:17:25.331717 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ebf9da-630c-46a2-8b7f-10b49e5cb846 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:25.333541 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ebf9da-630c-46a2-8b7f-10b49e5cb846 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:25.352963 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 992/1995] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:17:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:17:25.356378 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9db793-46f1-40a8-9a5d-4486040f5bb6 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:25.366602 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9db793-46f1-40a8-9a5d-4486040f5bb6 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:25.372087 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959e191-f78b-428d-860e-3c012a3ea984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:25.372777 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959e191-f78b-428d-860e-3c012a3ea984 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:25.373095 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959e191-f78b-428d-860e-3c012a3ea984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:25.373376 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959e191-f78b-428d-860e-3c012a3ea984 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:25.374098 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8959e191-f78b-428d-860e-3c012a3ea984 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:25.376105 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8959e191-f78b-428d-860e-3c012a3ea984 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:25.404360 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1004/1996] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:17:25 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:25.408895 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ebf9da-630c-46a2-8b7f-10b49e5cb846 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:25.420056 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ebf9da-630c-46a2-8b7f-10b49e5cb846 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:25.444302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:25.487580 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2199bf-47c2-47a6-abaf-fc48603204b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:25.488278 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2199bf-47c2-47a6-abaf-fc48603204b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:25.488558 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2199bf-47c2-47a6-abaf-fc48603204b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:25.488832 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2199bf-47c2-47a6-abaf-fc48603204b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:25.489584 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2199bf-47c2-47a6-abaf-fc48603204b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:25.491514 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2199bf-47c2-47a6-abaf-fc48603204b0 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:25.514626 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 993/1997] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:17:25 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:25.518902 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959e191-f78b-428d-860e-3c012a3ea984 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:25.530921 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8959e191-f78b-428d-860e-3c012a3ea984 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:25.536849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6282b026-016b-4d3a-be86-1a6939018584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:25.537577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6282b026-016b-4d3a-be86-1a6939018584 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:25.537862 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6282b026-016b-4d3a-be86-1a6939018584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:25.538135 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6282b026-016b-4d3a-be86-1a6939018584 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:25.538884 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6282b026-016b-4d3a-be86-1a6939018584 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:25.540993 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6282b026-016b-4d3a-be86-1a6939018584 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:25.552944 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1005/1998] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:25.557329 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2199bf-47c2-47a6-abaf-fc48603204b0 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:25.568051 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2199bf-47c2-47a6-abaf-fc48603204b0 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:25.573588 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1183f37-72ae-4099-98a3-0e2c9be6bb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:25.574406 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1183f37-72ae-4099-98a3-0e2c9be6bb77 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:25.574694 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1183f37-72ae-4099-98a3-0e2c9be6bb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:25.574977 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1183f37-72ae-4099-98a3-0e2c9be6bb77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:25.575725 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1183f37-72ae-4099-98a3-0e2c9be6bb77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:25.577509 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1183f37-72ae-4099-98a3-0e2c9be6bb77 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:25.589686 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 994/1999] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:25.594287 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6282b026-016b-4d3a-be86-1a6939018584 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:25.605583 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6282b026-016b-4d3a-be86-1a6939018584 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:25.610623 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4e3ec-03b4-440d-bb2a-1806451569e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:25.611566 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4e3ec-03b4-440d-bb2a-1806451569e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:25.611838 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4e3ec-03b4-440d-bb2a-1806451569e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:25.612107 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4e3ec-03b4-440d-bb2a-1806451569e3 admin admin] PATH_INFO: `/v3/projects/6a18caba069f421e9fa5456e73b527f8/users/75518e5d574541cb847d23385e8adc74/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:25.625425 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:25.630135 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4e3ec-03b4-440d-bb2a-1806451569e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a18caba069f421e9fa5456e73b527f8, user_id=75518e5d574541cb847d23385e8adc74, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:25.633697 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4e3ec-03b4-440d-bb2a-1806451569e3 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:25.648679 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1006/2000] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:17:25 2022] PUT /identity/v3/projects/6a18caba069f421e9fa5456e73b527f8/users/75518e5d574541cb847d23385e8adc74/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:25.660068 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae425a13-c073-4a75-a7ba-f9dd9bf20c4f None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:25.660634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae425a13-c073-4a75-a7ba-f9dd9bf20c4f None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:25.660634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae425a13-c073-4a75-a7ba-f9dd9bf20c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:25.689211 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-ae425a13-c073-4a75-a7ba-f9dd9bf20c4f None None] MFA Rules not processed for user `75518e5d574541cb847d23385e8adc74`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:25.690428 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-ae425a13-c073-4a75-a7ba-f9dd9bf20c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:25.762654 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 995/2001] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:25.774173 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4e3ec-03b4-440d-bb2a-1806451569e3 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:25.776167 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-2ac4e3ec-03b4-440d-bb2a-1806451569e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:25.853032 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4e3ec-03b4-440d-bb2a-1806451569e3 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:25.859046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841261d8-c7ff-4fc3-aaf4-086b99c1649f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:25.859694 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841261d8-c7ff-4fc3-aaf4-086b99c1649f service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:25.859969 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841261d8-c7ff-4fc3-aaf4-086b99c1649f service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:25.860237 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841261d8-c7ff-4fc3-aaf4-086b99c1649f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:25.869082 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841261d8-c7ff-4fc3-aaf4-086b99c1649f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:25.870683 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841261d8-c7ff-4fc3-aaf4-086b99c1649f service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:25.879926 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1007/2002] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:33.041215 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae425a13-c073-4a75-a7ba-f9dd9bf20c4f None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:33.042927 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-ae425a13-c073-4a75-a7ba-f9dd9bf20c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:33.128374 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae425a13-c073-4a75-a7ba-f9dd9bf20c4f None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:33.134624 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306f0a86-ce6b-45ee-86b9-203d4282a463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:33.135619 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306f0a86-ce6b-45ee-86b9-203d4282a463 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:33.135888 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306f0a86-ce6b-45ee-86b9-203d4282a463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:33.136164 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306f0a86-ce6b-45ee-86b9-203d4282a463 admin admin] PATH_INFO: `/v3/users/c9eb3e21e293403993d9093ed85926f4` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:33.137465 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306f0a86-ce6b-45ee-86b9-203d4282a463 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9eb3e21e293403993d9093ed85926f4)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:33.139436 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306f0a86-ce6b-45ee-86b9-203d4282a463 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:33.183727 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:33.195799 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-306f0a86-ce6b-45ee-86b9-203d4282a463 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9eb3e21e293403993d9093ed85926f4'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:33.203154 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-306f0a86-ce6b-45ee-86b9-203d4282a463 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9eb3e21e293403993d9093ed85926f4'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:33.204565 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-306f0a86-ce6b-45ee-86b9-203d4282a463 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9eb3e21e293403993d9093ed85926f4'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:33.204565 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-306f0a86-ce6b-45ee-86b9-203d4282a463 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9eb3e21e293403993d9093ed85926f4'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:33.215846 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-306f0a86-ce6b-45ee-86b9-203d4282a463 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9eb3e21e293403993d9093ed85926f4'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:33.239069 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 996/2003] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:17:33 2022] DELETE /identity/v3/users/c9eb3e21e293403993d9093ed85926f4 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:33.244773 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2bfd03-ffef-4080-93e2-d1aa1e21840f None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:33.245431 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2bfd03-ffef-4080-93e2-d1aa1e21840f None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:33.245431 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2bfd03-ffef-4080-93e2-d1aa1e21840f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:33.267003 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-de2bfd03-ffef-4080-93e2-d1aa1e21840f None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:33.268255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-de2bfd03-ffef-4080-93e2-d1aa1e21840f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:33.333411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1008/2004] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:33.344285 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306f0a86-ce6b-45ee-86b9-203d4282a463 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:33.345817 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-306f0a86-ce6b-45ee-86b9-203d4282a463 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:33.418105 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306f0a86-ce6b-45ee-86b9-203d4282a463 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:33.422313 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0477cfa0-db71-4c6a-8a0d-ded20438164f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:33.422989 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477cfa0-db71-4c6a-8a0d-ded20438164f service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:33.423333 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477cfa0-db71-4c6a-8a0d-ded20438164f service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:33.423841 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477cfa0-db71-4c6a-8a0d-ded20438164f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:33.435620 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0477cfa0-db71-4c6a-8a0d-ded20438164f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:33.437623 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0477cfa0-db71-4c6a-8a0d-ded20438164f service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:33.453114 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 997/2005] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:17:33.749963 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2bfd03-ffef-4080-93e2-d1aa1e21840f None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:33.752393 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-de2bfd03-ffef-4080-93e2-d1aa1e21840f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:33.773551 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2bfd03-ffef-4080-93e2-d1aa1e21840f None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:33.778896 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae946d72-5dfc-46ab-8df1-aac67bccf128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:33.779616 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae946d72-5dfc-46ab-8df1-aac67bccf128 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:33.779902 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae946d72-5dfc-46ab-8df1-aac67bccf128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:33.780178 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae946d72-5dfc-46ab-8df1-aac67bccf128 admin admin] PATH_INFO: `/v3/projects/6b53f279afe9432a87aed0df2b4272a4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:33.781906 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae946d72-5dfc-46ab-8df1-aac67bccf128 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b53f279afe9432a87aed0df2b4272a4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:33.783868 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae946d72-5dfc-46ab-8df1-aac67bccf128 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:33.832412 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ae946d72-5dfc-46ab-8df1-aac67bccf128 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b53f279afe9432a87aed0df2b4272a4', 'request_id': 'req-ae946d72-5dfc-46ab-8df1-aac67bccf128'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:33.839483 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ae946d72-5dfc-46ab-8df1-aac67bccf128 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b53f279afe9432a87aed0df2b4272a4', 'request_id': 'req-ae946d72-5dfc-46ab-8df1-aac67bccf128'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:33.848973 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ae946d72-5dfc-46ab-8df1-aac67bccf128 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b53f279afe9432a87aed0df2b4272a4', 'request_id': 'req-ae946d72-5dfc-46ab-8df1-aac67bccf128'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:33.856028 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:33.861488 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ae946d72-5dfc-46ab-8df1-aac67bccf128 admin admin] The token cache is being invalidate because project 6b53f279afe9432a87aed0df2b4272a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:17:33.861793 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ae946d72-5dfc-46ab-8df1-aac67bccf128 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:33.863275 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ae946d72-5dfc-46ab-8df1-aac67bccf128 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:33.865119 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1009/2006] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:17:33 2022] DELETE /identity/v3/projects/6b53f279afe9432a87aed0df2b4272a4 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:33.870968 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e504ed5-dd82-4af1-96ea-66a1392f299f None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:33.871259 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e504ed5-dd82-4af1-96ea-66a1392f299f None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:33.871539 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e504ed5-dd82-4af1-96ea-66a1392f299f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:33.890996 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-6e504ed5-dd82-4af1-96ea-66a1392f299f None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:33.892220 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-6e504ed5-dd82-4af1-96ea-66a1392f299f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:33.962105 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 998/2007] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:33.971583 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70e75e-3f00-4661-b6f8-20df0d29a40b None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:33.971934 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70e75e-3f00-4661-b6f8-20df0d29a40b None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:33.972231 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70e75e-3f00-4661-b6f8-20df0d29a40b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:33.993820 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-af70e75e-3f00-4661-b6f8-20df0d29a40b None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:33.995262 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-af70e75e-3f00-4661-b6f8-20df0d29a40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:34.006833 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1010/2008] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:34.011444 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e504ed5-dd82-4af1-96ea-66a1392f299f None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:34.024005 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e504ed5-dd82-4af1-96ea-66a1392f299f None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:34.031895 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6543fd91-f0a7-4a9a-a715-dca96c7d3b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:34.033027 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6543fd91-f0a7-4a9a-a715-dca96c7d3b13 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:34.034328 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6543fd91-f0a7-4a9a-a715-dca96c7d3b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:34.034765 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6543fd91-f0a7-4a9a-a715-dca96c7d3b13 admin admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:34.035716 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6543fd91-f0a7-4a9a-a715-dca96c7d3b13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:17:34.036140 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6543fd91-f0a7-4a9a-a715-dca96c7d3b13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:34.038294 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6543fd91-f0a7-4a9a-a715-dca96c7d3b13 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:34.054057 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 999/2009] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:17:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:17:34.058917 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af70e75e-3f00-4661-b6f8-20df0d29a40b None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:34.069103 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af70e75e-3f00-4661-b6f8-20df0d29a40b None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:34.075802 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf5bdd-ca4e-4851-8b82-2b8245b339c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:34.076472 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf5bdd-ca4e-4851-8b82-2b8245b339c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:34.076753 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf5bdd-ca4e-4851-8b82-2b8245b339c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:34.077067 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf5bdd-ca4e-4851-8b82-2b8245b339c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:34.077776 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaf5bdd-ca4e-4851-8b82-2b8245b339c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:34.079810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaf5bdd-ca4e-4851-8b82-2b8245b339c6 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:34.108527 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1011/2010] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:17:34 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:34.112696 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6543fd91-f0a7-4a9a-a715-dca96c7d3b13 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:34.123752 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6543fd91-f0a7-4a9a-a715-dca96c7d3b13 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:34.167680 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:34.189338 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e4a677-5fd7-4117-b6b7-12afe888a7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:34.190287 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4a677-5fd7-4117-b6b7-12afe888a7bc admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:34.190738 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4a677-5fd7-4117-b6b7-12afe888a7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:34.190923 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4a677-5fd7-4117-b6b7-12afe888a7bc admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:34.191694 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e4a677-5fd7-4117-b6b7-12afe888a7bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:34.193696 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e4a677-5fd7-4117-b6b7-12afe888a7bc admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:34.225207 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1000/2011] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:17:34 2022] POST /identity/v3/users => generated 286 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:34.229948 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf5bdd-ca4e-4851-8b82-2b8245b339c6 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:34.240742 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf5bdd-ca4e-4851-8b82-2b8245b339c6 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:34.246842 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aecb2e-0d6d-498a-aeee-f7f1ff0285a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:34.247530 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aecb2e-0d6d-498a-aeee-f7f1ff0285a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:34.247810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aecb2e-0d6d-498a-aeee-f7f1ff0285a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:34.248085 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aecb2e-0d6d-498a-aeee-f7f1ff0285a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:34.248859 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aecb2e-0d6d-498a-aeee-f7f1ff0285a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:34.250537 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aecb2e-0d6d-498a-aeee-f7f1ff0285a8 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:34.262048 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1012/2012] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:34.267668 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e4a677-5fd7-4117-b6b7-12afe888a7bc admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:34.279295 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e4a677-5fd7-4117-b6b7-12afe888a7bc admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:34.284605 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99196730-ec96-4cb4-abe6-ff9d660677ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:34.285338 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99196730-ec96-4cb4-abe6-ff9d660677ed admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:34.285627 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99196730-ec96-4cb4-abe6-ff9d660677ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:34.285900 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99196730-ec96-4cb4-abe6-ff9d660677ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:34.286648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99196730-ec96-4cb4-abe6-ff9d660677ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:34.288262 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99196730-ec96-4cb4-abe6-ff9d660677ed admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:34.300347 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1001/2013] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:34.305609 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aecb2e-0d6d-498a-aeee-f7f1ff0285a8 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:34.319903 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aecb2e-0d6d-498a-aeee-f7f1ff0285a8 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:34.325116 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4d6ad-3c36-478d-88f7-cc33381272ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:34.326013 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4d6ad-3c36-478d-88f7-cc33381272ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:34.326281 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4d6ad-3c36-478d-88f7-cc33381272ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:34.326552 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4d6ad-3c36-478d-88f7-cc33381272ff admin admin] PATH_INFO: `/v3/projects/f484976e63ce44daa10eadd2a9d6cc76/users/4e8428365f1148a2934dd4596deeb43a/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:34.342409 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec4d6ad-3c36-478d-88f7-cc33381272ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=f484976e63ce44daa10eadd2a9d6cc76, user_id=4e8428365f1148a2934dd4596deeb43a, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:34.345147 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec4d6ad-3c36-478d-88f7-cc33381272ff admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:34.359616 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1013/2014] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:17:34 2022] PUT /identity/v3/projects/f484976e63ce44daa10eadd2a9d6cc76/users/4e8428365f1148a2934dd4596deeb43a/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:34.365364 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e1357-99de-4c73-a6b2-bb3da12c4d63 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:34.365730 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e1357-99de-4c73-a6b2-bb3da12c4d63 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:34.366015 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e1357-99de-4c73-a6b2-bb3da12c4d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:34.385147 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-ea9e1357-99de-4c73-a6b2-bb3da12c4d63 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:34.386438 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-ea9e1357-99de-4c73-a6b2-bb3da12c4d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:34.451346 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1002/2015] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:34.461032 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4d6ad-3c36-478d-88f7-cc33381272ff admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:34.464475 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-1ec4d6ad-3c36-478d-88f7-cc33381272ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:34.521994 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4d6ad-3c36-478d-88f7-cc33381272ff admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:34.527047 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed24c22-5adf-4eee-b507-edd396a165ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:34.527896 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed24c22-5adf-4eee-b507-edd396a165ee service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:34.528175 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed24c22-5adf-4eee-b507-edd396a165ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:34.528446 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed24c22-5adf-4eee-b507-edd396a165ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:34.535197 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e1357-99de-4c73-a6b2-bb3da12c4d63 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:34.536725 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-ea9e1357-99de-4c73-a6b2-bb3da12c4d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:34.541161 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed24c22-5adf-4eee-b507-edd396a165ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:34.542767 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed24c22-5adf-4eee-b507-edd396a165ee service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:34.552213 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e1357-99de-4c73-a6b2-bb3da12c4d63 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:34.553911 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1014/2016] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:34.557562 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a260f344-2759-495d-95be-1e7d1a36a594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:34.558829 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a260f344-2759-495d-95be-1e7d1a36a594 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:34.559113 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a260f344-2759-495d-95be-1e7d1a36a594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:34.559412 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a260f344-2759-495d-95be-1e7d1a36a594 admin admin] PATH_INFO: `/v3/users/10262ff17c7f476a8a4ae14048c82c18` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:34.560834 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a260f344-2759-495d-95be-1e7d1a36a594 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10262ff17c7f476a8a4ae14048c82c18)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:34.562894 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a260f344-2759-495d-95be-1e7d1a36a594 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:34.625464 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-a260f344-2759-495d-95be-1e7d1a36a594 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10262ff17c7f476a8a4ae14048c82c18'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:34.636790 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-a260f344-2759-495d-95be-1e7d1a36a594 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10262ff17c7f476a8a4ae14048c82c18'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:34.637688 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-a260f344-2759-495d-95be-1e7d1a36a594 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10262ff17c7f476a8a4ae14048c82c18'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:34.638484 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-a260f344-2759-495d-95be-1e7d1a36a594 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10262ff17c7f476a8a4ae14048c82c18'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:34.646908 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-a260f344-2759-495d-95be-1e7d1a36a594 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10262ff17c7f476a8a4ae14048c82c18'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:34.669412 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1003/2017] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:17:34 2022] DELETE /identity/v3/users/10262ff17c7f476a8a4ae14048c82c18 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:17:34.673997 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290db37-d26f-4fee-a184-6c5675b63638 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:34.674380 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290db37-d26f-4fee-a184-6c5675b63638 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:34.675204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290db37-d26f-4fee-a184-6c5675b63638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:34.698124 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-6290db37-d26f-4fee-a184-6c5675b63638 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:34.699750 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-6290db37-d26f-4fee-a184-6c5675b63638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:34.784770 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1015/2018] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:34.794033 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a260f344-2759-495d-95be-1e7d1a36a594 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:34.796034 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-a260f344-2759-495d-95be-1e7d1a36a594 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:34.842829 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:34.875362 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a260f344-2759-495d-95be-1e7d1a36a594 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:34.879473 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e500a7-4e9e-4b14-ab3b-634804f921be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:34.880400 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e500a7-4e9e-4b14-ab3b-634804f921be service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:34.881074 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e500a7-4e9e-4b14-ab3b-634804f921be service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:34.881565 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e500a7-4e9e-4b14-ab3b-634804f921be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:34.893821 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e500a7-4e9e-4b14-ab3b-634804f921be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:34.895964 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e500a7-4e9e-4b14-ab3b-634804f921be service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:34.910419 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1004/2019] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:35.250148 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6290db37-d26f-4fee-a184-6c5675b63638 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:35.251781 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-6290db37-d26f-4fee-a184-6c5675b63638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:35.265187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6290db37-d26f-4fee-a184-6c5675b63638 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:35.270409 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404998ea-c3fa-42c1-ba22-03d624d73596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:35.271358 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404998ea-c3fa-42c1-ba22-03d624d73596 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:35.271623 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404998ea-c3fa-42c1-ba22-03d624d73596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:35.272081 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404998ea-c3fa-42c1-ba22-03d624d73596 admin admin] PATH_INFO: `/v3/users/6be5074fcecb45918bc904d9baa63aa5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:35.273368 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404998ea-c3fa-42c1-ba22-03d624d73596 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6be5074fcecb45918bc904d9baa63aa5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:35.275604 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404998ea-c3fa-42c1-ba22-03d624d73596 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:35.313530 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:35.330588 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-404998ea-c3fa-42c1-ba22-03d624d73596 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6be5074fcecb45918bc904d9baa63aa5'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:35.341139 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-404998ea-c3fa-42c1-ba22-03d624d73596 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6be5074fcecb45918bc904d9baa63aa5'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:35.341960 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-404998ea-c3fa-42c1-ba22-03d624d73596 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6be5074fcecb45918bc904d9baa63aa5'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:35.343012 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-404998ea-c3fa-42c1-ba22-03d624d73596 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6be5074fcecb45918bc904d9baa63aa5'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:35.353639 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-404998ea-c3fa-42c1-ba22-03d624d73596 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6be5074fcecb45918bc904d9baa63aa5'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:35.394125 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1016/2020] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:17:35 2022] DELETE /identity/v3/users/6be5074fcecb45918bc904d9baa63aa5 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:35.397396 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a65b3-b19e-45bc-944b-06ac6d0eb962 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:35.398020 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a65b3-b19e-45bc-944b-06ac6d0eb962 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:35.398020 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a65b3-b19e-45bc-944b-06ac6d0eb962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:35.418089 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-5d3a65b3-b19e-45bc-944b-06ac6d0eb962 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:35.419334 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-5d3a65b3-b19e-45bc-944b-06ac6d0eb962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:35.484969 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404998ea-c3fa-42c1-ba22-03d624d73596 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:35.486822 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-404998ea-c3fa-42c1-ba22-03d624d73596 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:35.502558 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1005/2021] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:35.513104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a65b3-b19e-45bc-944b-06ac6d0eb962 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:35.514507 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-5d3a65b3-b19e-45bc-944b-06ac6d0eb962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:35.568892 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404998ea-c3fa-42c1-ba22-03d624d73596 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:35.575297 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0cb81c-c626-4b1a-8a58-b27ff022214b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:35.576245 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0cb81c-c626-4b1a-8a58-b27ff022214b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:35.576670 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0cb81c-c626-4b1a-8a58-b27ff022214b admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:35.577012 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0cb81c-c626-4b1a-8a58-b27ff022214b admin admin] PATH_INFO: `/v3/users/75518e5d574541cb847d23385e8adc74` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:35.578757 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0cb81c-c626-4b1a-8a58-b27ff022214b admin admin] RBAC: Authorizing `identity:delete_user(user_id=75518e5d574541cb847d23385e8adc74)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:35.581173 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0cb81c-c626-4b1a-8a58-b27ff022214b admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:35.589070 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a65b3-b19e-45bc-944b-06ac6d0eb962 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:35.594435 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b7387d-eb76-4f5c-8664-685a2b9c4783 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:35.595211 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7387d-eb76-4f5c-8664-685a2b9c4783 service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:35.595567 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7387d-eb76-4f5c-8664-685a2b9c4783 service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:35.595992 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7387d-eb76-4f5c-8664-685a2b9c4783 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:35.608797 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b7387d-eb76-4f5c-8664-685a2b9c4783 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:35.610707 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b7387d-eb76-4f5c-8664-685a2b9c4783 service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:35.623695 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1006/2022] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:35.658325 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-aa0cb81c-c626-4b1a-8a58-b27ff022214b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75518e5d574541cb847d23385e8adc74'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:35.668552 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-aa0cb81c-c626-4b1a-8a58-b27ff022214b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75518e5d574541cb847d23385e8adc74'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:35.670197 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-aa0cb81c-c626-4b1a-8a58-b27ff022214b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75518e5d574541cb847d23385e8adc74'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:35.670936 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-aa0cb81c-c626-4b1a-8a58-b27ff022214b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75518e5d574541cb847d23385e8adc74'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:35.684793 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-aa0cb81c-c626-4b1a-8a58-b27ff022214b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75518e5d574541cb847d23385e8adc74'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:35.720336 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1017/2023] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:17:35 2022] DELETE /identity/v3/users/75518e5d574541cb847d23385e8adc74 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:35.725344 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57867aef-df20-43a2-a2d5-1a017087791b None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:35.725726 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57867aef-df20-43a2-a2d5-1a017087791b None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:35.726090 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57867aef-df20-43a2-a2d5-1a017087791b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:35.751703 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-57867aef-df20-43a2-a2d5-1a017087791b None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:35.754081 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-57867aef-df20-43a2-a2d5-1a017087791b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:35.860302 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0cb81c-c626-4b1a-8a58-b27ff022214b admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:35.861815 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-aa0cb81c-c626-4b1a-8a58-b27ff022214b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:35.916784 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:35.926536 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1007/2024] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:17:35.947742 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57867aef-df20-43a2-a2d5-1a017087791b None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:35.949632 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-57867aef-df20-43a2-a2d5-1a017087791b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:35.975020 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0cb81c-c626-4b1a-8a58-b27ff022214b admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:35.981400 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3290c60-160f-46f5-9ab6-1047de1e6355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:35.984471 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3290c60-160f-46f5-9ab6-1047de1e6355 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:35.984471 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3290c60-160f-46f5-9ab6-1047de1e6355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:35.984721 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3290c60-160f-46f5-9ab6-1047de1e6355 admin admin] PATH_INFO: `/v3/projects/4833ce998ff047cfb90a2fc82abf277e` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:35.986923 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3290c60-160f-46f5-9ab6-1047de1e6355 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4833ce998ff047cfb90a2fc82abf277e)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:35.990304 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3290c60-160f-46f5-9ab6-1047de1e6355 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:35.997825 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:36.086939 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57867aef-df20-43a2-a2d5-1a017087791b None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:36.089374 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b3290c60-160f-46f5-9ab6-1047de1e6355 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4833ce998ff047cfb90a2fc82abf277e', 'request_id': 'req-b3290c60-160f-46f5-9ab6-1047de1e6355'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:36.102917 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b3290c60-160f-46f5-9ab6-1047de1e6355 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4833ce998ff047cfb90a2fc82abf277e', 'request_id': 'req-b3290c60-160f-46f5-9ab6-1047de1e6355'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:36.135676 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b3290c60-160f-46f5-9ab6-1047de1e6355 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4833ce998ff047cfb90a2fc82abf277e', 'request_id': 'req-b3290c60-160f-46f5-9ab6-1047de1e6355'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:36.171627 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b3290c60-160f-46f5-9ab6-1047de1e6355 admin admin] The token cache is being invalidate because project 4833ce998ff047cfb90a2fc82abf277e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:17:36.172125 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b3290c60-160f-46f5-9ab6-1047de1e6355 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:36.176237 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b3290c60-160f-46f5-9ab6-1047de1e6355 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:36.178726 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1018/2025] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:17:35 2022] DELETE /identity/v3/projects/4833ce998ff047cfb90a2fc82abf277e => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:17:36.179246 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e85e406-7c72-4660-a9c6-6e778a5ad4a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:36.180197 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85e406-7c72-4660-a9c6-6e778a5ad4a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:36.180315 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3290c60-160f-46f5-9ab6-1047de1e6355 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:36.180531 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85e406-7c72-4660-a9c6-6e778a5ad4a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:36.180812 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85e406-7c72-4660-a9c6-6e778a5ad4a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:36.182987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-2e85e406-7c72-4660-a9c6-6e778a5ad4a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:36.183934 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-b3290c60-160f-46f5-9ab6-1047de1e6355 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:36.313778 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3290c60-160f-46f5-9ab6-1047de1e6355 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:36.320774 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a329a0bc-0278-4c39-9538-ec633855f510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:36.322865 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329a0bc-0278-4c39-9538-ec633855f510 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:36.323196 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329a0bc-0278-4c39-9538-ec633855f510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:36.323478 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329a0bc-0278-4c39-9538-ec633855f510 admin admin] PATH_INFO: `/v3/projects/6212e66c6a8642879fcda0a07f4aa68a` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:36.325642 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a329a0bc-0278-4c39-9538-ec633855f510 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6212e66c6a8642879fcda0a07f4aa68a)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:36.328335 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a329a0bc-0278-4c39-9538-ec633855f510 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:36.340547 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e85e406-7c72-4660-a9c6-6e778a5ad4a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:36.342699 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e85e406-7c72-4660-a9c6-6e778a5ad4a1 service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:36.371225 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1008/2026] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:36.373639 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfaf43-38f2-4387-9945-f76012bc2ca8 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:36.373928 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfaf43-38f2-4387-9945-f76012bc2ca8 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:36.374589 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfaf43-38f2-4387-9945-f76012bc2ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:36.403879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a329a0bc-0278-4c39-9538-ec633855f510 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6212e66c6a8642879fcda0a07f4aa68a', 'request_id': 'req-a329a0bc-0278-4c39-9538-ec633855f510'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:36.405126 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-9bcfaf43-38f2-4387-9945-f76012bc2ca8 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:36.406256 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-9bcfaf43-38f2-4387-9945-f76012bc2ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:36.419097 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a329a0bc-0278-4c39-9538-ec633855f510 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6212e66c6a8642879fcda0a07f4aa68a', 'request_id': 'req-a329a0bc-0278-4c39-9538-ec633855f510'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:36.446009 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a329a0bc-0278-4c39-9538-ec633855f510 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6212e66c6a8642879fcda0a07f4aa68a', 'request_id': 'req-a329a0bc-0278-4c39-9538-ec633855f510'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:36.448812 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:36.465894 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a329a0bc-0278-4c39-9538-ec633855f510 admin admin] The token cache is being invalidate because project 6212e66c6a8642879fcda0a07f4aa68a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:17:36.466453 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a329a0bc-0278-4c39-9538-ec633855f510 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:36.467479 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a329a0bc-0278-4c39-9538-ec633855f510 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:36.469742 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1019/2027] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:17:36 2022] DELETE /identity/v3/projects/6212e66c6a8642879fcda0a07f4aa68a => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:36.503881 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1009/2028] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:36.514387 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b38b1a-79fa-4d53-abfc-6d5cbaf50e9e None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:36.514742 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b38b1a-79fa-4d53-abfc-6d5cbaf50e9e None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:36.514970 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b38b1a-79fa-4d53-abfc-6d5cbaf50e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:36.563095 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-04b38b1a-79fa-4d53-abfc-6d5cbaf50e9e None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:36.565154 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-04b38b1a-79fa-4d53-abfc-6d5cbaf50e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:36.593641 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1020/2029] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:36.595652 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcfaf43-38f2-4387-9945-f76012bc2ca8 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:36.620687 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcfaf43-38f2-4387-9945-f76012bc2ca8 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:36.630000 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a964bdd2-bab2-43d7-b5a5-dd4c601c5abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:36.631067 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964bdd2-bab2-43d7-b5a5-dd4c601c5abc admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:36.631704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964bdd2-bab2-43d7-b5a5-dd4c601c5abc admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:36.633364 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964bdd2-bab2-43d7-b5a5-dd4c601c5abc admin admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:36.634646 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a964bdd2-bab2-43d7-b5a5-dd4c601c5abc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:17:36.635225 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a964bdd2-bab2-43d7-b5a5-dd4c601c5abc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:36.637934 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a964bdd2-bab2-43d7-b5a5-dd4c601c5abc admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:36.659597 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1010/2030] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:17:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:17:36.663237 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b38b1a-79fa-4d53-abfc-6d5cbaf50e9e None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:36.686531 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b38b1a-79fa-4d53-abfc-6d5cbaf50e9e None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:36.704657 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c708ae-c71a-48f8-8919-5d7d8a3ab44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:36.706378 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c708ae-c71a-48f8-8919-5d7d8a3ab44c admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:36.707013 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c708ae-c71a-48f8-8919-5d7d8a3ab44c admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:36.707630 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c708ae-c71a-48f8-8919-5d7d8a3ab44c admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:36.708490 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c708ae-c71a-48f8-8919-5d7d8a3ab44c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:36.711672 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c708ae-c71a-48f8-8919-5d7d8a3ab44c admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:36.748371 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1021/2031] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:17:36 2022] POST /identity/v3/projects => generated 389 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:17:36.749921 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a964bdd2-bab2-43d7-b5a5-dd4c601c5abc admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:36.761174 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a964bdd2-bab2-43d7-b5a5-dd4c601c5abc admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:36.779346 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:36.868917 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4639e15-7349-4a9e-9d26-3b57aecc732b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:36.869652 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4639e15-7349-4a9e-9d26-3b57aecc732b admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:36.869956 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4639e15-7349-4a9e-9d26-3b57aecc732b admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:36.870208 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4639e15-7349-4a9e-9d26-3b57aecc732b admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:36.870965 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4639e15-7349-4a9e-9d26-3b57aecc732b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:36.872970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4639e15-7349-4a9e-9d26-3b57aecc732b admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:36.917994 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1011/2032] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:17:36 2022] POST /identity/v3/users => generated 295 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:36.919564 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c708ae-c71a-48f8-8919-5d7d8a3ab44c admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:36.931849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c708ae-c71a-48f8-8919-5d7d8a3ab44c admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:36.938123 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad42d7e-f412-4df4-b501-81163ea1f9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:36.938440 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad42d7e-f412-4df4-b501-81163ea1f9c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:36.938713 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad42d7e-f412-4df4-b501-81163ea1f9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:36.938988 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad42d7e-f412-4df4-b501-81163ea1f9c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:36.939885 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad42d7e-f412-4df4-b501-81163ea1f9c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:36.941749 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad42d7e-f412-4df4-b501-81163ea1f9c0 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:36.958851 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1022/2033] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:36.960442 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4639e15-7349-4a9e-9d26-3b57aecc732b admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:36.973303 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4639e15-7349-4a9e-9d26-3b57aecc732b admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:36.979987 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a823111a-ddbe-4561-b590-c92c6383ff77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:36.983574 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a823111a-ddbe-4561-b590-c92c6383ff77 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:36.983574 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a823111a-ddbe-4561-b590-c92c6383ff77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:36.983574 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a823111a-ddbe-4561-b590-c92c6383ff77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:36.983574 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a823111a-ddbe-4561-b590-c92c6383ff77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:36.984040 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a823111a-ddbe-4561-b590-c92c6383ff77 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:36.984915 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad42d7e-f412-4df4-b501-81163ea1f9c0 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:36.987293 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-aad42d7e-f412-4df4-b501-81163ea1f9c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:36.996978 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1012/2034] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:37.000620 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a823111a-ddbe-4561-b590-c92c6383ff77 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:37.009047 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad42d7e-f412-4df4-b501-81163ea1f9c0 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:37.016481 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a823111a-ddbe-4561-b590-c92c6383ff77 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:37.022568 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6143a7be-e076-415e-859f-296c1c5f1dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:37.025485 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6143a7be-e076-415e-859f-296c1c5f1dbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:37.026774 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6143a7be-e076-415e-859f-296c1c5f1dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:37.027076 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6143a7be-e076-415e-859f-296c1c5f1dbd admin admin] PATH_INFO: `/v3/projects/6a18caba069f421e9fa5456e73b527f8` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:37.029146 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6143a7be-e076-415e-859f-296c1c5f1dbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a18caba069f421e9fa5456e73b527f8)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:37.031185 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6143a7be-e076-415e-859f-296c1c5f1dbd admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:37.032474 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f94ca43-dd69-4307-bfa4-5a6d8f38d007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:37.033826 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94ca43-dd69-4307-bfa4-5a6d8f38d007 admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:37.034290 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94ca43-dd69-4307-bfa4-5a6d8f38d007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:37.034927 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94ca43-dd69-4307-bfa4-5a6d8f38d007 admin admin] PATH_INFO: `/v3/projects/04bdba8864c8481180acedc32712fafe/users/975c6ffce82e4c5db815695dfe3c235b/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:37.057527 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f94ca43-dd69-4307-bfa4-5a6d8f38d007 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04bdba8864c8481180acedc32712fafe, user_id=975c6ffce82e4c5db815695dfe3c235b, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:37.063056 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f94ca43-dd69-4307-bfa4-5a6d8f38d007 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:37.106923 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1013/2035] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:17:36 2022] PUT /identity/v3/projects/04bdba8864c8481180acedc32712fafe/users/975c6ffce82e4c5db815695dfe3c235b/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:17:37.118649 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260f856-99b5-4277-badb-4d4658352cde None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:37.118940 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260f856-99b5-4277-badb-4d4658352cde None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:37.119219 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260f856-99b5-4277-badb-4d4658352cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:37.132981 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6143a7be-e076-415e-859f-296c1c5f1dbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a18caba069f421e9fa5456e73b527f8', 'request_id': 'req-6143a7be-e076-415e-859f-296c1c5f1dbd'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:37.140569 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6143a7be-e076-415e-859f-296c1c5f1dbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a18caba069f421e9fa5456e73b527f8', 'request_id': 'req-6143a7be-e076-415e-859f-296c1c5f1dbd'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:37.150058 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6143a7be-e076-415e-859f-296c1c5f1dbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a18caba069f421e9fa5456e73b527f8', 'request_id': 'req-6143a7be-e076-415e-859f-296c1c5f1dbd'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:37.150339 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-e260f856-99b5-4277-badb-4d4658352cde None None] MFA Rules not processed for user `975c6ffce82e4c5db815695dfe3c235b`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:37.151657 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-e260f856-99b5-4277-badb-4d4658352cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:37.172871 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6143a7be-e076-415e-859f-296c1c5f1dbd admin admin] The token cache is being invalidate because project 6a18caba069f421e9fa5456e73b527f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:17:37.173206 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6143a7be-e076-415e-859f-296c1c5f1dbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:37.176099 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-6143a7be-e076-415e-859f-296c1c5f1dbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:37.179062 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1023/2036] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:17:36 2022] DELETE /identity/v3/projects/6a18caba069f421e9fa5456e73b527f8 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:37.184508 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa6cf9-7f16-42e7-b94d-0095ff70d63f None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:37.185055 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa6cf9-7f16-42e7-b94d-0095ff70d63f None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:37.185202 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa6cf9-7f16-42e7-b94d-0095ff70d63f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:37.212960 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-e3fa6cf9-7f16-42e7-b94d-0095ff70d63f None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:37.214325 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-e3fa6cf9-7f16-42e7-b94d-0095ff70d63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:37.284986 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1014/2037] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:37.298423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e260f856-99b5-4277-badb-4d4658352cde None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:37.300321 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-e260f856-99b5-4277-badb-4d4658352cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:37.311315 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1024/2038] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:37.313614 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6940e5b-6aa4-487b-aa64-b3d5a7cf85ce None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:37.314160 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6940e5b-6aa4-487b-aa64-b3d5a7cf85ce None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:37.314708 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6940e5b-6aa4-487b-aa64-b3d5a7cf85ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:37.341368 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-d6940e5b-6aa4-487b-aa64-b3d5a7cf85ce None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:37.343371 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d6940e5b-6aa4-487b-aa64-b3d5a7cf85ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:37.352824 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1025/2039] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:37.357490 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6940e5b-6aa4-487b-aa64-b3d5a7cf85ce None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:37.370890 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6940e5b-6aa4-487b-aa64-b3d5a7cf85ce None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:37.383418 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1f5839-81ce-4c74-9550-c1d6c3ede527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:37.385214 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f5839-81ce-4c74-9550-c1d6c3ede527 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:37.385969 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f5839-81ce-4c74-9550-c1d6c3ede527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:37.386424 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f5839-81ce-4c74-9550-c1d6c3ede527 admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:37.387739 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1f5839-81ce-4c74-9550-c1d6c3ede527 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:17:37.388427 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1f5839-81ce-4c74-9550-c1d6c3ede527 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:37.390511 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1f5839-81ce-4c74-9550-c1d6c3ede527 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:37.397833 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e260f856-99b5-4277-badb-4d4658352cde None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:37.401917 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260dee9c-7b0a-44f9-bacb-5b39b5d56dcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:37.402596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260dee9c-7b0a-44f9-bacb-5b39b5d56dcf service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:37.402877 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260dee9c-7b0a-44f9-bacb-5b39b5d56dcf service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:37.403151 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260dee9c-7b0a-44f9-bacb-5b39b5d56dcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:37.410859 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1026/2040] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:17:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:17:37.413301 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260dee9c-7b0a-44f9-bacb-5b39b5d56dcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:37.416131 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1f5839-81ce-4c74-9550-c1d6c3ede527 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:37.419277 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260dee9c-7b0a-44f9-bacb-5b39b5d56dcf service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:37.429683 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:37.434630 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1f5839-81ce-4c74-9550-c1d6c3ede527 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:37.440131 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6be966-f09d-433e-8c64-a7829aa12380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:37.441007 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6be966-f09d-433e-8c64-a7829aa12380 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:37.441323 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6be966-f09d-433e-8c64-a7829aa12380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:37.441644 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6be966-f09d-433e-8c64-a7829aa12380 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:37.442578 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6be966-f09d-433e-8c64-a7829aa12380 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:37.444912 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6be966-f09d-433e-8c64-a7829aa12380 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:37.456909 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1015/2041] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:37.458647 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:37.484974 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1027/2042] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:17:37 2022] POST /identity/v3/projects => generated 373 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:37.489441 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260dee9c-7b0a-44f9-bacb-5b39b5d56dcf service cinder] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:37.506838 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260dee9c-7b0a-44f9-bacb-5b39b5d56dcf service cinder] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:37.588617 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764a65e-a432-4f5c-99b5-c4f7ec8eaeb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:37.589674 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764a65e-a432-4f5c-99b5-c4f7ec8eaeb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:37.589948 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764a65e-a432-4f5c-99b5-c4f7ec8eaeb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:37.590215 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764a65e-a432-4f5c-99b5-c4f7ec8eaeb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:37.591046 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9764a65e-a432-4f5c-99b5-c4f7ec8eaeb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:37.593509 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9764a65e-a432-4f5c-99b5-c4f7ec8eaeb0 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:37.625641 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1016/2043] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:17:37 2022] POST /identity/v3/users => generated 287 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:17:37.631959 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6be966-f09d-433e-8c64-a7829aa12380 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:37.650861 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6be966-f09d-433e-8c64-a7829aa12380 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:37.658607 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c2e3df-e729-4396-b25a-826d745c05cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:37.659501 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c2e3df-e729-4396-b25a-826d745c05cd admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:37.660654 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c2e3df-e729-4396-b25a-826d745c05cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:37.661586 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c2e3df-e729-4396-b25a-826d745c05cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:37.662379 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c2e3df-e729-4396-b25a-826d745c05cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:37.664021 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c2e3df-e729-4396-b25a-826d745c05cd admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:37.676714 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1028/2044] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:37.681075 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764a65e-a432-4f5c-99b5-c4f7ec8eaeb0 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:37.696564 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764a65e-a432-4f5c-99b5-c4f7ec8eaeb0 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:37.705724 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b59dc-d152-46bf-a9cf-6f41abb6697d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:37.706488 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b59dc-d152-46bf-a9cf-6f41abb6697d admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:37.706838 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b59dc-d152-46bf-a9cf-6f41abb6697d admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:37.707178 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b59dc-d152-46bf-a9cf-6f41abb6697d admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:37.707994 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35b59dc-d152-46bf-a9cf-6f41abb6697d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:37.709799 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35b59dc-d152-46bf-a9cf-6f41abb6697d admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:37.723584 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1017/2045] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:37.728442 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c2e3df-e729-4396-b25a-826d745c05cd admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:37.740271 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c2e3df-e729-4396-b25a-826d745c05cd admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:37.748405 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e85f0dc-7411-4102-9329-f5c27d571647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:37.749406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85f0dc-7411-4102-9329-f5c27d571647 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:37.749688 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85f0dc-7411-4102-9329-f5c27d571647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:37.749963 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85f0dc-7411-4102-9329-f5c27d571647 admin admin] PATH_INFO: `/v3/projects/bdb74511f0614aa59d5b44c0f677ab69/users/3575b258461f489b8f97a7aa85628b17/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:37.775456 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e85f0dc-7411-4102-9329-f5c27d571647 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdb74511f0614aa59d5b44c0f677ab69, user_id=3575b258461f489b8f97a7aa85628b17, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:37.778564 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e85f0dc-7411-4102-9329-f5c27d571647 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:37.799272 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1029/2046] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:17:37 2022] PUT /identity/v3/projects/bdb74511f0614aa59d5b44c0f677ab69/users/3575b258461f489b8f97a7aa85628b17/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:37.820091 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47641bf-7038-4521-b95f-701a8e3285b3 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:37.820471 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47641bf-7038-4521-b95f-701a8e3285b3 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:37.820809 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47641bf-7038-4521-b95f-701a8e3285b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:37.883913 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-d47641bf-7038-4521-b95f-701a8e3285b3 None None] MFA Rules not processed for user `3575b258461f489b8f97a7aa85628b17`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:37.885221 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-d47641bf-7038-4521-b95f-701a8e3285b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:37.984758 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1018/2047] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:37.989348 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e85f0dc-7411-4102-9329-f5c27d571647 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:38.090941 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e85f0dc-7411-4102-9329-f5c27d571647 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:38.096615 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1062473b-0be1-4cd3-8ec9-f1a30e098ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:38.097394 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062473b-0be1-4cd3-8ec9-f1a30e098ae1 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:38.097679 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062473b-0be1-4cd3-8ec9-f1a30e098ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:38.097961 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062473b-0be1-4cd3-8ec9-f1a30e098ae1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:38.098690 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1062473b-0be1-4cd3-8ec9-f1a30e098ae1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:38.100958 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1062473b-0be1-4cd3-8ec9-f1a30e098ae1 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:38.139568 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1030/2048] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:17:37 2022] POST /identity/v3/projects => generated 373 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:38.140524 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47641bf-7038-4521-b95f-701a8e3285b3 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:38.153079 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47641bf-7038-4521-b95f-701a8e3285b3 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:38.227955 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39770cf2-8584-4933-b514-210ba8b35be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:38.228080 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39770cf2-8584-4933-b514-210ba8b35be1 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:38.228452 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39770cf2-8584-4933-b514-210ba8b35be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:38.228758 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39770cf2-8584-4933-b514-210ba8b35be1 admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:38.229518 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39770cf2-8584-4933-b514-210ba8b35be1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:38.231811 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39770cf2-8584-4933-b514-210ba8b35be1 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:38.249335 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:38.260274 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1019/2049] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:17:38 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:17:38.264639 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1062473b-0be1-4cd3-8ec9-f1a30e098ae1 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:38.278414 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1062473b-0be1-4cd3-8ec9-f1a30e098ae1 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:38.285369 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd4f3a-3573-41a9-a33f-a33be0767c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:38.286564 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd4f3a-3573-41a9-a33f-a33be0767c14 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:38.287031 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd4f3a-3573-41a9-a33f-a33be0767c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:38.287477 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd4f3a-3573-41a9-a33f-a33be0767c14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:38.288611 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd4f3a-3573-41a9-a33f-a33be0767c14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:38.291501 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd4f3a-3573-41a9-a33f-a33be0767c14 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:38.311155 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1031/2050] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:38.314151 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39770cf2-8584-4933-b514-210ba8b35be1 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:38.328691 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39770cf2-8584-4933-b514-210ba8b35be1 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:38.334641 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a515e3a-e2f1-4f46-a521-0e501dd13942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:38.335696 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a515e3a-e2f1-4f46-a521-0e501dd13942 admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:38.336174 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a515e3a-e2f1-4f46-a521-0e501dd13942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:38.336720 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a515e3a-e2f1-4f46-a521-0e501dd13942 admin admin] PATH_INFO: `/v3/projects/abb6e6e9ef864b9bb9375998d1c30064/users/44eb4af717da4f4ca382bfca4339ac1f/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:38.358605 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a515e3a-e2f1-4f46-a521-0e501dd13942 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abb6e6e9ef864b9bb9375998d1c30064, user_id=44eb4af717da4f4ca382bfca4339ac1f, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:38.361627 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a515e3a-e2f1-4f46-a521-0e501dd13942 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:38.385327 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1020/2051] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:17:38 2022] PUT /identity/v3/projects/abb6e6e9ef864b9bb9375998d1c30064/users/44eb4af717da4f4ca382bfca4339ac1f/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:38.387241 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd4f3a-3573-41a9-a33f-a33be0767c14 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:38.399610 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd4f3a-3573-41a9-a33f-a33be0767c14 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:38.470378 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d8b8ef-851e-4b5a-8377-dc9678b82212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:38.471489 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8b8ef-851e-4b5a-8377-dc9678b82212 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:38.471820 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8b8ef-851e-4b5a-8377-dc9678b82212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:38.472170 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8b8ef-851e-4b5a-8377-dc9678b82212 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:38.473065 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d8b8ef-851e-4b5a-8377-dc9678b82212 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:38.475105 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d8b8ef-851e-4b5a-8377-dc9678b82212 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:38.491073 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1032/2052] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:38.492295 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a515e3a-e2f1-4f46-a521-0e501dd13942 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:38.504326 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a515e3a-e2f1-4f46-a521-0e501dd13942 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:38.511579 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7159d8-0f8c-4b9a-af68-60cb8e1841df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:38.512641 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7159d8-0f8c-4b9a-af68-60cb8e1841df admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:38.513025 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7159d8-0f8c-4b9a-af68-60cb8e1841df admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:38.513311 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7159d8-0f8c-4b9a-af68-60cb8e1841df admin admin] PATH_INFO: `/v3/projects/abb6e6e9ef864b9bb9375998d1c30064/users/44eb4af717da4f4ca382bfca4339ac1f/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:38.515625 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7159d8-0f8c-4b9a-af68-60cb8e1841df admin admin] RBAC: Authorizing `identity:create_grant(project_id=abb6e6e9ef864b9bb9375998d1c30064, user_id=44eb4af717da4f4ca382bfca4339ac1f, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:38.518453 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7159d8-0f8c-4b9a-af68-60cb8e1841df admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:38.530392 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1021/2053] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:17:38 2022] PUT /identity/v3/projects/abb6e6e9ef864b9bb9375998d1c30064/users/44eb4af717da4f4ca382bfca4339ac1f/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:38.542904 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8db46-2934-450d-8b55-40769e32124d None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:38.543216 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8db46-2934-450d-8b55-40769e32124d None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:38.543471 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8db46-2934-450d-8b55-40769e32124d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:38.576831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-5aa8db46-2934-450d-8b55-40769e32124d None None] MFA Rules not processed for user `44eb4af717da4f4ca382bfca4339ac1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:38.578599 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-5aa8db46-2934-450d-8b55-40769e32124d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:38.594579 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:38.677637 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1033/2054] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:38 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:38.694309 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7159d8-0f8c-4b9a-af68-60cb8e1841df admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:38.813220 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7159d8-0f8c-4b9a-af68-60cb8e1841df admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:38.823166 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7a79d5-b9cc-491f-9c39-d0ed439e407b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:38.824221 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a79d5-b9cc-491f-9c39-d0ed439e407b service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:38.824512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a79d5-b9cc-491f-9c39-d0ed439e407b service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:38.824794 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a79d5-b9cc-491f-9c39-d0ed439e407b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:38.858486 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7a79d5-b9cc-491f-9c39-d0ed439e407b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:38.860128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7a79d5-b9cc-491f-9c39-d0ed439e407b service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:38.874814 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1022/2055] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:38 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:17:39.417257 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa8db46-2934-450d-8b55-40769e32124d None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:39.513884 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa8db46-2934-450d-8b55-40769e32124d None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:39.519482 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f824b28d-5e51-48c7-b681-0f288a99629b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:39.520539 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f824b28d-5e51-48c7-b681-0f288a99629b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:39.521173 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f824b28d-5e51-48c7-b681-0f288a99629b admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:39.521589 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f824b28d-5e51-48c7-b681-0f288a99629b admin admin] PATH_INFO: `/v3/users/3575b258461f489b8f97a7aa85628b17` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:39.523250 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f824b28d-5e51-48c7-b681-0f288a99629b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3575b258461f489b8f97a7aa85628b17)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:39.526588 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f824b28d-5e51-48c7-b681-0f288a99629b admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:39.618627 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-f824b28d-5e51-48c7-b681-0f288a99629b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3575b258461f489b8f97a7aa85628b17'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:39.630454 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-f824b28d-5e51-48c7-b681-0f288a99629b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3575b258461f489b8f97a7aa85628b17'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:39.631358 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-f824b28d-5e51-48c7-b681-0f288a99629b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3575b258461f489b8f97a7aa85628b17'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:39.632077 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-f824b28d-5e51-48c7-b681-0f288a99629b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3575b258461f489b8f97a7aa85628b17'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:39.643002 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-f824b28d-5e51-48c7-b681-0f288a99629b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3575b258461f489b8f97a7aa85628b17'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:39.675426 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1034/2056] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:17:39 2022] DELETE /identity/v3/users/3575b258461f489b8f97a7aa85628b17 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:39.676839 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff08092-743c-47d5-a4ae-b1ddcf383a0a None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:39.677345 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff08092-743c-47d5-a4ae-b1ddcf383a0a None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:39.677438 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff08092-743c-47d5-a4ae-b1ddcf383a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:39.708163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-cff08092-743c-47d5-a4ae-b1ddcf383a0a None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:39.709429 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-cff08092-743c-47d5-a4ae-b1ddcf383a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:39.780897 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1023/2057] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:17:39.790626 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f824b28d-5e51-48c7-b681-0f288a99629b admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:39.792214 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-f824b28d-5e51-48c7-b681-0f288a99629b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:39.836543 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:39.885976 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f824b28d-5e51-48c7-b681-0f288a99629b admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:39.889616 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d6222-87b8-49a8-b153-e8726a2fddfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:39.890665 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d6222-87b8-49a8-b153-e8726a2fddfb service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:39.890800 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d6222-87b8-49a8-b153-e8726a2fddfb service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:39.891008 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d6222-87b8-49a8-b153-e8726a2fddfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:39.902914 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729d6222-87b8-49a8-b153-e8726a2fddfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:39.904424 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729d6222-87b8-49a8-b153-e8726a2fddfb service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:39.916250 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1035/2058] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:40.482306 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4affacb7-aba6-45c7-a3fa-1e95c8a87e61 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:40.482850 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4affacb7-aba6-45c7-a3fa-1e95c8a87e61 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:40.483343 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4affacb7-aba6-45c7-a3fa-1e95c8a87e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:40.499617 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d6222-87b8-49a8-b153-e8726a2fddfb service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:40.502293 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-729d6222-87b8-49a8-b153-e8726a2fddfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:40.520343 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d6222-87b8-49a8-b153-e8726a2fddfb service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:40.520554 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-4affacb7-aba6-45c7-a3fa-1e95c8a87e61 None None] MFA Rules not processed for user `4e8428365f1148a2934dd4596deeb43a`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:40.521798 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-4affacb7-aba6-45c7-a3fa-1e95c8a87e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:40.527198 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8e81a-70a9-4d42-abf0-9eff54cffe50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:40.528527 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8e81a-70a9-4d42-abf0-9eff54cffe50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:40.528956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8e81a-70a9-4d42-abf0-9eff54cffe50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:40.529467 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8e81a-70a9-4d42-abf0-9eff54cffe50 admin admin] PATH_INFO: `/v3/projects/bdb74511f0614aa59d5b44c0f677ab69` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:40.532079 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8e81a-70a9-4d42-abf0-9eff54cffe50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdb74511f0614aa59d5b44c0f677ab69)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:40.534333 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8e81a-70a9-4d42-abf0-9eff54cffe50 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:40.590022 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:40.591530 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-55f8e81a-70a9-4d42-abf0-9eff54cffe50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdb74511f0614aa59d5b44c0f677ab69', 'request_id': 'req-55f8e81a-70a9-4d42-abf0-9eff54cffe50'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:40.599511 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-55f8e81a-70a9-4d42-abf0-9eff54cffe50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdb74511f0614aa59d5b44c0f677ab69', 'request_id': 'req-55f8e81a-70a9-4d42-abf0-9eff54cffe50'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:40.610349 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-55f8e81a-70a9-4d42-abf0-9eff54cffe50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdb74511f0614aa59d5b44c0f677ab69', 'request_id': 'req-55f8e81a-70a9-4d42-abf0-9eff54cffe50'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:40.614648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1024/2059] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:17:40.616371 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4affacb7-aba6-45c7-a3fa-1e95c8a87e61 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:40.618425 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-4affacb7-aba6-45c7-a3fa-1e95c8a87e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:40.625303 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-55f8e81a-70a9-4d42-abf0-9eff54cffe50 admin admin] The token cache is being invalidate because project bdb74511f0614aa59d5b44c0f677ab69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:17:40.625599 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-55f8e81a-70a9-4d42-abf0-9eff54cffe50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:40.626237 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-55f8e81a-70a9-4d42-abf0-9eff54cffe50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:40.627819 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1036/2060] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:17:40 2022] DELETE /identity/v3/projects/bdb74511f0614aa59d5b44c0f677ab69 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:17:40.631719 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8e81a-70a9-4d42-abf0-9eff54cffe50 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:40.633372 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4affacb7-aba6-45c7-a3fa-1e95c8a87e61 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:40.633980 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-55f8e81a-70a9-4d42-abf0-9eff54cffe50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:40.698401 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1082d33a-903a-4986-8eab-b013cd3dc268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:40.699114 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1082d33a-903a-4986-8eab-b013cd3dc268 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:40.699399 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1082d33a-903a-4986-8eab-b013cd3dc268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:40.699672 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1082d33a-903a-4986-8eab-b013cd3dc268 admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:40.700377 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1082d33a-903a-4986-8eab-b013cd3dc268 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:40.702835 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1082d33a-903a-4986-8eab-b013cd3dc268 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:40.706974 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8e81a-70a9-4d42-abf0-9eff54cffe50 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:40.712622 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bf45f3-506c-4af7-b735-84908479cf87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:40.713597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf45f3-506c-4af7-b735-84908479cf87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:40.713884 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf45f3-506c-4af7-b735-84908479cf87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:40.714163 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf45f3-506c-4af7-b735-84908479cf87 admin admin] PATH_INFO: `/v3/users/44eb4af717da4f4ca382bfca4339ac1f` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:40.715704 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bf45f3-506c-4af7-b735-84908479cf87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44eb4af717da4f4ca382bfca4339ac1f)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:40.717735 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bf45f3-506c-4af7-b735-84908479cf87 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:40.735168 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1025/2061] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:17:40 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:17:40.739034 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1082d33a-903a-4986-8eab-b013cd3dc268 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:40.740675 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-1082d33a-903a-4986-8eab-b013cd3dc268 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:40.774933 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-24bf45f3-506c-4af7-b735-84908479cf87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44eb4af717da4f4ca382bfca4339ac1f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:40.783132 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-24bf45f3-506c-4af7-b735-84908479cf87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44eb4af717da4f4ca382bfca4339ac1f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:40.785470 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-24bf45f3-506c-4af7-b735-84908479cf87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44eb4af717da4f4ca382bfca4339ac1f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:40.786188 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-24bf45f3-506c-4af7-b735-84908479cf87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44eb4af717da4f4ca382bfca4339ac1f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:40.790146 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:40.801255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-24bf45f3-506c-4af7-b735-84908479cf87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44eb4af717da4f4ca382bfca4339ac1f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:40.821580 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1082d33a-903a-4986-8eab-b013cd3dc268 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:40.827088 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7638a275-c538-4662-9f65-68d067090657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:40.827897 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638a275-c538-4662-9f65-68d067090657 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:40.828375 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638a275-c538-4662-9f65-68d067090657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:40.828658 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638a275-c538-4662-9f65-68d067090657 admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:40.829566 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7638a275-c538-4662-9f65-68d067090657 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:40.830724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1037/2062] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:17:40 2022] DELETE /identity/v3/users/44eb4af717da4f4ca382bfca4339ac1f => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:40.831839 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7638a275-c538-4662-9f65-68d067090657 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:40.857694 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1026/2063] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:17:40 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:17:40.861709 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bf45f3-506c-4af7-b735-84908479cf87 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:40.863306 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-24bf45f3-506c-4af7-b735-84908479cf87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:40.943516 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bf45f3-506c-4af7-b735-84908479cf87 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:40.949293 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c898f079-a431-4458-94b8-51802dc0d2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:40.949970 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898f079-a431-4458-94b8-51802dc0d2bf admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:40.950238 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898f079-a431-4458-94b8-51802dc0d2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:40.950511 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898f079-a431-4458-94b8-51802dc0d2bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:40.951393 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c898f079-a431-4458-94b8-51802dc0d2bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:40.953213 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c898f079-a431-4458-94b8-51802dc0d2bf admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:40.965217 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1038/2064] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:40 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:40.969473 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7638a275-c538-4662-9f65-68d067090657 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:40.980982 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7638a275-c538-4662-9f65-68d067090657 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:40.985765 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ddfbfb-7f4c-4b7a-a14d-4635b73828c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:40.986759 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ddfbfb-7f4c-4b7a-a14d-4635b73828c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:40.987040 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ddfbfb-7f4c-4b7a-a14d-4635b73828c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:40.987316 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ddfbfb-7f4c-4b7a-a14d-4635b73828c5 admin admin] PATH_INFO: `/v3/projects/d2507516008649269ca7932a008d7db3/users/fce6f6160c2e4021ba36f0deb035446e/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:41.004477 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ddfbfb-7f4c-4b7a-a14d-4635b73828c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2507516008649269ca7932a008d7db3, user_id=fce6f6160c2e4021ba36f0deb035446e, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:41.007432 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ddfbfb-7f4c-4b7a-a14d-4635b73828c5 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:41.025913 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1027/2065] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:17:40 2022] PUT /identity/v3/projects/d2507516008649269ca7932a008d7db3/users/fce6f6160c2e4021ba36f0deb035446e/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:17:41.030354 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c898f079-a431-4458-94b8-51802dc0d2bf admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:41.041014 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c898f079-a431-4458-94b8-51802dc0d2bf admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:41.101645 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f6277-7a41-4d2d-bf53-50b2bbce48bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:41.102374 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f6277-7a41-4d2d-bf53-50b2bbce48bc admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:41.102668 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f6277-7a41-4d2d-bf53-50b2bbce48bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:41.102927 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f6277-7a41-4d2d-bf53-50b2bbce48bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:41.103801 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862f6277-7a41-4d2d-bf53-50b2bbce48bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:41.105451 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862f6277-7a41-4d2d-bf53-50b2bbce48bc admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:41.116660 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1039/2066] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:41 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:17:41.121088 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ddfbfb-7f4c-4b7a-a14d-4635b73828c5 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:41.131506 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ddfbfb-7f4c-4b7a-a14d-4635b73828c5 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:41.137393 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de834276-84b0-43d6-904e-78a7f9f85ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:41.138445 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de834276-84b0-43d6-904e-78a7f9f85ea3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:41.138888 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de834276-84b0-43d6-904e-78a7f9f85ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:41.139324 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de834276-84b0-43d6-904e-78a7f9f85ea3 admin admin] PATH_INFO: `/v3/projects/d2507516008649269ca7932a008d7db3/users/fce6f6160c2e4021ba36f0deb035446e/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:41.142205 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de834276-84b0-43d6-904e-78a7f9f85ea3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2507516008649269ca7932a008d7db3, user_id=fce6f6160c2e4021ba36f0deb035446e, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:41.145652 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de834276-84b0-43d6-904e-78a7f9f85ea3 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:41.156817 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f6277-7a41-4d2d-bf53-50b2bbce48bc admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:41.159590 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-862f6277-7a41-4d2d-bf53-50b2bbce48bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:41.163326 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1028/2067] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:17:41 2022] PUT /identity/v3/projects/d2507516008649269ca7932a008d7db3/users/fce6f6160c2e4021ba36f0deb035446e/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:41.184326 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f6277-7a41-4d2d-bf53-50b2bbce48bc admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:41.250668 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611f2e1-4c21-496c-9d86-ebfeff78be0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:41.252021 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611f2e1-4c21-496c-9d86-ebfeff78be0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:41.252407 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611f2e1-4c21-496c-9d86-ebfeff78be0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:41.252760 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611f2e1-4c21-496c-9d86-ebfeff78be0a admin admin] PATH_INFO: `/v3/projects/abb6e6e9ef864b9bb9375998d1c30064` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:41.254367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b611f2e1-4c21-496c-9d86-ebfeff78be0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=abb6e6e9ef864b9bb9375998d1c30064)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:41.258016 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b611f2e1-4c21-496c-9d86-ebfeff78be0a admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:41.271262 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:41.318966 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b611f2e1-4c21-496c-9d86-ebfeff78be0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abb6e6e9ef864b9bb9375998d1c30064', 'request_id': 'req-b611f2e1-4c21-496c-9d86-ebfeff78be0a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:41.328495 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b611f2e1-4c21-496c-9d86-ebfeff78be0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abb6e6e9ef864b9bb9375998d1c30064', 'request_id': 'req-b611f2e1-4c21-496c-9d86-ebfeff78be0a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:41.339351 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b611f2e1-4c21-496c-9d86-ebfeff78be0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abb6e6e9ef864b9bb9375998d1c30064', 'request_id': 'req-b611f2e1-4c21-496c-9d86-ebfeff78be0a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:41.354336 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b611f2e1-4c21-496c-9d86-ebfeff78be0a admin admin] The token cache is being invalidate because project abb6e6e9ef864b9bb9375998d1c30064 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:17:41.354819 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b611f2e1-4c21-496c-9d86-ebfeff78be0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:41.356710 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b611f2e1-4c21-496c-9d86-ebfeff78be0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:41.361365 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1040/2068] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:17:41 2022] DELETE /identity/v3/projects/abb6e6e9ef864b9bb9375998d1c30064 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:41.366989 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6bb7c-0781-4292-b389-1adbbff07003 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:41.367243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6bb7c-0781-4292-b389-1adbbff07003 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:41.367388 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6bb7c-0781-4292-b389-1adbbff07003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:41.415145 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-b5c6bb7c-0781-4292-b389-1adbbff07003 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:41.422125 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-b5c6bb7c-0781-4292-b389-1adbbff07003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:41.484286 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:41.569518 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1029/2069] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:41.584968 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c0a56-3ac2-46c0-9dff-0349c804ca08 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:41.585568 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c0a56-3ac2-46c0-9dff-0349c804ca08 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:41.585568 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c0a56-3ac2-46c0-9dff-0349c804ca08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:41.610470 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-f64c0a56-3ac2-46c0-9dff-0349c804ca08 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:41.614243 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-f64c0a56-3ac2-46c0-9dff-0349c804ca08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:41.628975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1041/2070] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:41.633462 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c6bb7c-0781-4292-b389-1adbbff07003 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:41.644506 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c6bb7c-0781-4292-b389-1adbbff07003 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:41.653610 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6e780-1706-4fb1-b427-c6965f640d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:41.654759 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6e780-1706-4fb1-b427-c6965f640d1c admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:41.655261 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6e780-1706-4fb1-b427-c6965f640d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:41.655769 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6e780-1706-4fb1-b427-c6965f640d1c admin admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:41.657674 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d6e780-1706-4fb1-b427-c6965f640d1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:17:41.658194 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d6e780-1706-4fb1-b427-c6965f640d1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:41.660792 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d6e780-1706-4fb1-b427-c6965f640d1c admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:41.677787 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1030/2071] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:17:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:17:41.682525 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64c0a56-3ac2-46c0-9dff-0349c804ca08 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:41.695592 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64c0a56-3ac2-46c0-9dff-0349c804ca08 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:41.703420 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4924ba2-939d-4942-b769-ffbd1a3851d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:41.704172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4924ba2-939d-4942-b769-ffbd1a3851d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:41.704457 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4924ba2-939d-4942-b769-ffbd1a3851d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:41.704748 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4924ba2-939d-4942-b769-ffbd1a3851d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:41.705763 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4924ba2-939d-4942-b769-ffbd1a3851d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:41.709570 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4924ba2-939d-4942-b769-ffbd1a3851d2 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:41.749417 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1042/2072] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:17:41 2022] POST /identity/v3/projects => generated 369 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:41.750740 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6e780-1706-4fb1-b427-c6965f640d1c admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:41.761675 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6e780-1706-4fb1-b427-c6965f640d1c admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:41.832052 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf6049b-14e1-4c6a-9b0f-3a9980f37742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:41.832996 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf6049b-14e1-4c6a-9b0f-3a9980f37742 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:41.833345 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf6049b-14e1-4c6a-9b0f-3a9980f37742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:41.833686 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf6049b-14e1-4c6a-9b0f-3a9980f37742 admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:41.834465 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf6049b-14e1-4c6a-9b0f-3a9980f37742 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:41.836549 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf6049b-14e1-4c6a-9b0f-3a9980f37742 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:41.863768 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1031/2073] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:17:41 2022] POST /identity/v3/users => generated 285 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:17:41.866629 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4924ba2-939d-4942-b769-ffbd1a3851d2 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:41.881633 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4924ba2-939d-4942-b769-ffbd1a3851d2 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:41.894738 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814da674-5c9b-4121-919d-3770752826b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:41.896352 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814da674-5c9b-4121-919d-3770752826b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:41.896797 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814da674-5c9b-4121-919d-3770752826b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:41.897096 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814da674-5c9b-4121-919d-3770752826b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:41.897986 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814da674-5c9b-4121-919d-3770752826b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:41.904410 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814da674-5c9b-4121-919d-3770752826b1 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:41.919906 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1043/2074] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:41 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:41.923990 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf6049b-14e1-4c6a-9b0f-3a9980f37742 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:41.939089 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf6049b-14e1-4c6a-9b0f-3a9980f37742 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:41.944545 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0c0cc-a66e-49d0-89e0-41819efc015b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:41.945689 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0c0cc-a66e-49d0-89e0-41819efc015b admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:41.945855 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0c0cc-a66e-49d0-89e0-41819efc015b admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:41.946124 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0c0cc-a66e-49d0-89e0-41819efc015b admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:41.947114 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db0c0cc-a66e-49d0-89e0-41819efc015b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:41.948872 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db0c0cc-a66e-49d0-89e0-41819efc015b admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:41.978522 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1032/2075] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:41 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:41.985115 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814da674-5c9b-4121-919d-3770752826b1 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:42.006421 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814da674-5c9b-4121-919d-3770752826b1 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:42.013797 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3495de-8130-4654-bb8f-9508cf22cc29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:42.015583 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3495de-8130-4654-bb8f-9508cf22cc29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:42.016090 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3495de-8130-4654-bb8f-9508cf22cc29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:42.017654 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3495de-8130-4654-bb8f-9508cf22cc29 admin admin] PATH_INFO: `/v3/projects/e18ebe094e6246c89ecf5dd4bf04a9e1/users/092e89f18e2f4861bef1950a1cd042b7/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:42.056731 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3495de-8130-4654-bb8f-9508cf22cc29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e18ebe094e6246c89ecf5dd4bf04a9e1, user_id=092e89f18e2f4861bef1950a1cd042b7, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:42.062248 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3495de-8130-4654-bb8f-9508cf22cc29 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:42.081348 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1044/2076] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:17:41 2022] PUT /identity/v3/projects/e18ebe094e6246c89ecf5dd4bf04a9e1/users/092e89f18e2f4861bef1950a1cd042b7/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:42.094234 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b8d16-bac8-4f5c-b6c1-8e6050c1135a None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:42.094555 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b8d16-bac8-4f5c-b6c1-8e6050c1135a None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:42.094976 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b8d16-bac8-4f5c-b6c1-8e6050c1135a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:42.136535 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-c69b8d16-bac8-4f5c-b6c1-8e6050c1135a None None] MFA Rules not processed for user `092e89f18e2f4861bef1950a1cd042b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:42.138266 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-c69b8d16-bac8-4f5c-b6c1-8e6050c1135a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:42.244118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1033/2077] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:42 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:42.248418 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3495de-8130-4654-bb8f-9508cf22cc29 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:42.361484 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3495de-8130-4654-bb8f-9508cf22cc29 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:42.369981 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b005963-9c86-4872-be49-17505465b300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:42.370712 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b005963-9c86-4872-be49-17505465b300 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:42.371729 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b005963-9c86-4872-be49-17505465b300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:42.372555 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b005963-9c86-4872-be49-17505465b300 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:42.372730 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b005963-9c86-4872-be49-17505465b300 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:42.375491 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b005963-9c86-4872-be49-17505465b300 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:42.410941 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1045/2078] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:17:42 2022] POST /identity/v3/projects => generated 367 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:17:42.415542 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b8d16-bac8-4f5c-b6c1-8e6050c1135a None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:42.449025 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b8d16-bac8-4f5c-b6c1-8e6050c1135a None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:42.502282 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:42.556878 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7051df2-3a47-447d-b2f5-518d46cb280a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:42.557717 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7051df2-3a47-447d-b2f5-518d46cb280a admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:42.558068 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7051df2-3a47-447d-b2f5-518d46cb280a admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:42.558402 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7051df2-3a47-447d-b2f5-518d46cb280a admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:42.559555 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7051df2-3a47-447d-b2f5-518d46cb280a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:42.562275 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7051df2-3a47-447d-b2f5-518d46cb280a admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:42.594825 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1034/2079] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:17:42 2022] POST /identity/v3/users => generated 290 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:17:42.602831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b005963-9c86-4872-be49-17505465b300 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:42.623905 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b005963-9c86-4872-be49-17505465b300 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:42.631630 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2656d1-56a9-4bdc-a03d-0ebc9a15fa7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:42.632046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2656d1-56a9-4bdc-a03d-0ebc9a15fa7d admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:42.632828 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2656d1-56a9-4bdc-a03d-0ebc9a15fa7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:42.633431 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2656d1-56a9-4bdc-a03d-0ebc9a15fa7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:42.634435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2656d1-56a9-4bdc-a03d-0ebc9a15fa7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:42.636968 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2656d1-56a9-4bdc-a03d-0ebc9a15fa7d admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:42.663467 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1046/2080] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:42 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:17:42.669741 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7051df2-3a47-447d-b2f5-518d46cb280a admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:42.684855 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7051df2-3a47-447d-b2f5-518d46cb280a admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:42.703163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7466689b-8434-4b17-bc9a-1eb8d9292a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:42.704382 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7466689b-8434-4b17-bc9a-1eb8d9292a7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:42.705250 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7466689b-8434-4b17-bc9a-1eb8d9292a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:42.705695 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7466689b-8434-4b17-bc9a-1eb8d9292a7c admin admin] PATH_INFO: `/v3/projects/bcbcea9a3c1b413789e497a320e2201e/users/d817a0ba7d814bfebeccdebf9ba471e2/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:42.739527 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7466689b-8434-4b17-bc9a-1eb8d9292a7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcbcea9a3c1b413789e497a320e2201e, user_id=d817a0ba7d814bfebeccdebf9ba471e2, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:42.743877 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7466689b-8434-4b17-bc9a-1eb8d9292a7c admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:42.770184 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1035/2081] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:17:42 2022] PUT /identity/v3/projects/bcbcea9a3c1b413789e497a320e2201e/users/d817a0ba7d814bfebeccdebf9ba471e2/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:17:42.777267 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2656d1-56a9-4bdc-a03d-0ebc9a15fa7d admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:42.851827 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2656d1-56a9-4bdc-a03d-0ebc9a15fa7d admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:42.875279 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:43.003767 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065f0a3c-a766-4d48-9315-61dcf98461dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:43.010123 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f0a3c-a766-4d48-9315-61dcf98461dd admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:43.010551 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f0a3c-a766-4d48-9315-61dcf98461dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:43.010794 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f0a3c-a766-4d48-9315-61dcf98461dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:43.011618 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065f0a3c-a766-4d48-9315-61dcf98461dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:43.014221 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065f0a3c-a766-4d48-9315-61dcf98461dd admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:43.035001 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1047/2082] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:42 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:43.039831 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7466689b-8434-4b17-bc9a-1eb8d9292a7c admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:43.057158 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7466689b-8434-4b17-bc9a-1eb8d9292a7c admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:43.078998 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b7a18f-7b7d-4adc-b8ad-48755058d47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:43.079998 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7a18f-7b7d-4adc-b8ad-48755058d47d admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:43.080208 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7a18f-7b7d-4adc-b8ad-48755058d47d admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:43.080777 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7a18f-7b7d-4adc-b8ad-48755058d47d admin admin] PATH_INFO: `/v3/projects/bcbcea9a3c1b413789e497a320e2201e/users/d817a0ba7d814bfebeccdebf9ba471e2/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:43.083088 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b7a18f-7b7d-4adc-b8ad-48755058d47d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcbcea9a3c1b413789e497a320e2201e, user_id=d817a0ba7d814bfebeccdebf9ba471e2, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:43.086685 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b7a18f-7b7d-4adc-b8ad-48755058d47d admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:43.104073 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1036/2083] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:17:43 2022] PUT /identity/v3/projects/bcbcea9a3c1b413789e497a320e2201e/users/d817a0ba7d814bfebeccdebf9ba471e2/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:43.117566 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b4acd-3085-45e5-b99a-7e919f2aa837 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:43.117903 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b4acd-3085-45e5-b99a-7e919f2aa837 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:43.118152 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b4acd-3085-45e5-b99a-7e919f2aa837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:43.166428 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-f42b4acd-3085-45e5-b99a-7e919f2aa837 None None] MFA Rules not processed for user `d817a0ba7d814bfebeccdebf9ba471e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:43.167787 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-f42b4acd-3085-45e5-b99a-7e919f2aa837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:43.303287 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1048/2084] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:43 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:43.315359 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b7a18f-7b7d-4adc-b8ad-48755058d47d admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:43.324873 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-35b7a18f-7b7d-4adc-b8ad-48755058d47d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:43.483714 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b7a18f-7b7d-4adc-b8ad-48755058d47d admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:43.489548 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f68145-0950-4820-96db-13571ec4dfea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:43.490324 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f68145-0950-4820-96db-13571ec4dfea service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:43.490678 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f68145-0950-4820-96db-13571ec4dfea service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:43.491045 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f68145-0950-4820-96db-13571ec4dfea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:43.511682 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f68145-0950-4820-96db-13571ec4dfea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:43.513492 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f68145-0950-4820-96db-13571ec4dfea service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:43.535038 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1037/2085] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:43 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:45.735010 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b4acd-3085-45e5-b99a-7e919f2aa837 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:45.826354 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42b4acd-3085-45e5-b99a-7e919f2aa837 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:45.832094 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f229fd-b1fd-4b48-bde1-ea1b694e02b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:45.833110 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f229fd-b1fd-4b48-bde1-ea1b694e02b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:45.833576 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f229fd-b1fd-4b48-bde1-ea1b694e02b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:45.833855 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f229fd-b1fd-4b48-bde1-ea1b694e02b5 admin admin] PATH_INFO: `/v3/users/092e89f18e2f4861bef1950a1cd042b7` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:45.835097 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f229fd-b1fd-4b48-bde1-ea1b694e02b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=092e89f18e2f4861bef1950a1cd042b7)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:45.838392 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f229fd-b1fd-4b48-bde1-ea1b694e02b5 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:45.907416 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:45.927555 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-60f229fd-b1fd-4b48-bde1-ea1b694e02b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '092e89f18e2f4861bef1950a1cd042b7'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:45.939050 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-60f229fd-b1fd-4b48-bde1-ea1b694e02b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '092e89f18e2f4861bef1950a1cd042b7'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:45.942152 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-60f229fd-b1fd-4b48-bde1-ea1b694e02b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '092e89f18e2f4861bef1950a1cd042b7'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:45.944756 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-60f229fd-b1fd-4b48-bde1-ea1b694e02b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '092e89f18e2f4861bef1950a1cd042b7'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:45.961243 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-60f229fd-b1fd-4b48-bde1-ea1b694e02b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '092e89f18e2f4861bef1950a1cd042b7'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:46.015040 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1049/2086] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:17:45 2022] DELETE /identity/v3/users/092e89f18e2f4861bef1950a1cd042b7 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:46.016821 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e02be-6d80-4b81-ae72-0ad58fa2e59e None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:46.017605 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e02be-6d80-4b81-ae72-0ad58fa2e59e None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:46.017605 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e02be-6d80-4b81-ae72-0ad58fa2e59e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:46.051092 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-4f6e02be-6d80-4b81-ae72-0ad58fa2e59e None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:46.052519 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-4f6e02be-6d80-4b81-ae72-0ad58fa2e59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:46.188368 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1038/2087] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:46.210971 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f229fd-b1fd-4b48-bde1-ea1b694e02b5 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:46.213521 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-60f229fd-b1fd-4b48-bde1-ea1b694e02b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:46.376052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f229fd-b1fd-4b48-bde1-ea1b694e02b5 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:46.391059 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c7748-86e3-4b3d-8869-61af41e81ba1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:46.392742 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c7748-86e3-4b3d-8869-61af41e81ba1 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:46.393377 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c7748-86e3-4b3d-8869-61af41e81ba1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:46.393856 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c7748-86e3-4b3d-8869-61af41e81ba1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:46.409347 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61c7748-86e3-4b3d-8869-61af41e81ba1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:46.411319 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61c7748-86e3-4b3d-8869-61af41e81ba1 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:46.424494 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1050/2088] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:47.046614 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6e02be-6d80-4b81-ae72-0ad58fa2e59e None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:47.048770 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-4f6e02be-6d80-4b81-ae72-0ad58fa2e59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:47.064794 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6e02be-6d80-4b81-ae72-0ad58fa2e59e None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:47.077365 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd14ceb4-e65f-49bf-9e03-6489d71c2549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:47.078287 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14ceb4-e65f-49bf-9e03-6489d71c2549 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:47.078738 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14ceb4-e65f-49bf-9e03-6489d71c2549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:47.079012 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14ceb4-e65f-49bf-9e03-6489d71c2549 admin admin] PATH_INFO: `/v3/projects/e18ebe094e6246c89ecf5dd4bf04a9e1` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:47.080201 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd14ceb4-e65f-49bf-9e03-6489d71c2549 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e18ebe094e6246c89ecf5dd4bf04a9e1)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:47.082749 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd14ceb4-e65f-49bf-9e03-6489d71c2549 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:47.105451 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:47.149327 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-bd14ceb4-e65f-49bf-9e03-6489d71c2549 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e18ebe094e6246c89ecf5dd4bf04a9e1', 'request_id': 'req-bd14ceb4-e65f-49bf-9e03-6489d71c2549'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:47.156742 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-bd14ceb4-e65f-49bf-9e03-6489d71c2549 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e18ebe094e6246c89ecf5dd4bf04a9e1', 'request_id': 'req-bd14ceb4-e65f-49bf-9e03-6489d71c2549'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:47.166911 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-bd14ceb4-e65f-49bf-9e03-6489d71c2549 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e18ebe094e6246c89ecf5dd4bf04a9e1', 'request_id': 'req-bd14ceb4-e65f-49bf-9e03-6489d71c2549'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:47.182043 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-bd14ceb4-e65f-49bf-9e03-6489d71c2549 admin admin] The token cache is being invalidate because project e18ebe094e6246c89ecf5dd4bf04a9e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:17:47.182285 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-bd14ceb4-e65f-49bf-9e03-6489d71c2549 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:47.182980 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-bd14ceb4-e65f-49bf-9e03-6489d71c2549 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:47.184764 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1039/2089] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:17:47 2022] DELETE /identity/v3/projects/e18ebe094e6246c89ecf5dd4bf04a9e1 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:47.188956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c7748-86e3-4b3d-8869-61af41e81ba1 service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:47.200972 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c7748-86e3-4b3d-8869-61af41e81ba1 service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:47.278026 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f63fbb0-4302-4bb7-9815-9c00d216e99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:47.278891 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f63fbb0-4302-4bb7-9815-9c00d216e99a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:47.279148 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f63fbb0-4302-4bb7-9815-9c00d216e99a admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:47.279789 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f63fbb0-4302-4bb7-9815-9c00d216e99a admin admin] PATH_INFO: `/v3/users/d817a0ba7d814bfebeccdebf9ba471e2` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:47.280617 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f63fbb0-4302-4bb7-9815-9c00d216e99a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d817a0ba7d814bfebeccdebf9ba471e2)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:47.283585 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f63fbb0-4302-4bb7-9815-9c00d216e99a admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:47.354911 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-3f63fbb0-4302-4bb7-9815-9c00d216e99a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd817a0ba7d814bfebeccdebf9ba471e2'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:47.361287 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:47.366210 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-3f63fbb0-4302-4bb7-9815-9c00d216e99a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd817a0ba7d814bfebeccdebf9ba471e2'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:47.368256 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-3f63fbb0-4302-4bb7-9815-9c00d216e99a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd817a0ba7d814bfebeccdebf9ba471e2'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:47.368997 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-3f63fbb0-4302-4bb7-9815-9c00d216e99a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd817a0ba7d814bfebeccdebf9ba471e2'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:47.381036 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-3f63fbb0-4302-4bb7-9815-9c00d216e99a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd817a0ba7d814bfebeccdebf9ba471e2'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:47.406733 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1051/2090] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:17:47 2022] DELETE /identity/v3/users/d817a0ba7d814bfebeccdebf9ba471e2 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:47.511478 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca62f6f-b4b1-404f-a4f9-069666b91ba0 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:47.512298 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca62f6f-b4b1-404f-a4f9-069666b91ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:47.512684 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca62f6f-b4b1-404f-a4f9-069666b91ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:47.572891 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-3ca62f6f-b4b1-404f-a4f9-069666b91ba0 None None] MFA Rules not processed for user `fce6f6160c2e4021ba36f0deb035446e`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:47.574210 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-3ca62f6f-b4b1-404f-a4f9-069666b91ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:47.706353 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1040/2091] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:47 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:47.721062 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f63fbb0-4302-4bb7-9815-9c00d216e99a admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:47.722764 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-3f63fbb0-4302-4bb7-9815-9c00d216e99a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:47.802723 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca62f6f-b4b1-404f-a4f9-069666b91ba0 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:47.804178 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-3ca62f6f-b4b1-404f-a4f9-069666b91ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:47.834962 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f63fbb0-4302-4bb7-9815-9c00d216e99a admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:47.847453 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc35a6-87ba-4388-885c-d28401a4cd73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:47.848271 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc35a6-87ba-4388-885c-d28401a4cd73 service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:47.848753 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc35a6-87ba-4388-885c-d28401a4cd73 service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:47.849068 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc35a6-87ba-4388-885c-d28401a4cd73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:47.851836 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-70bc35a6-87ba-4388-885c-d28401a4cd73 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:47.891767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca62f6f-b4b1-404f-a4f9-069666b91ba0 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:47.898567 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4450457c-27d6-4c7a-a6b1-9b7203f11408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:47.899562 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450457c-27d6-4c7a-a6b1-9b7203f11408 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:47.899780 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450457c-27d6-4c7a-a6b1-9b7203f11408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:47.900055 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450457c-27d6-4c7a-a6b1-9b7203f11408 admin admin] PATH_INFO: `/v3/projects/bcbcea9a3c1b413789e497a320e2201e` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:47.901345 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4450457c-27d6-4c7a-a6b1-9b7203f11408 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcbcea9a3c1b413789e497a320e2201e)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:47.903383 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4450457c-27d6-4c7a-a6b1-9b7203f11408 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:47.923818 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bc35a6-87ba-4388-885c-d28401a4cd73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:47.925500 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bc35a6-87ba-4388-885c-d28401a4cd73 service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:47.936110 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1052/2092] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:47.945813 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:47.964879 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-4450457c-27d6-4c7a-a6b1-9b7203f11408 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcbcea9a3c1b413789e497a320e2201e', 'request_id': 'req-4450457c-27d6-4c7a-a6b1-9b7203f11408'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:47.971554 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-4450457c-27d6-4c7a-a6b1-9b7203f11408 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcbcea9a3c1b413789e497a320e2201e', 'request_id': 'req-4450457c-27d6-4c7a-a6b1-9b7203f11408'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:47.982761 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-4450457c-27d6-4c7a-a6b1-9b7203f11408 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcbcea9a3c1b413789e497a320e2201e', 'request_id': 'req-4450457c-27d6-4c7a-a6b1-9b7203f11408'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:47.994393 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-4450457c-27d6-4c7a-a6b1-9b7203f11408 admin admin] The token cache is being invalidate because project bcbcea9a3c1b413789e497a320e2201e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:17:47.994691 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-4450457c-27d6-4c7a-a6b1-9b7203f11408 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:47.995351 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-4450457c-27d6-4c7a-a6b1-9b7203f11408 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:47.997036 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1041/2093] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:17:47 2022] DELETE /identity/v3/projects/bcbcea9a3c1b413789e497a320e2201e => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:48.001857 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33375436-90f6-400c-ba6c-cba7dc478276 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:48.002145 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33375436-90f6-400c-ba6c-cba7dc478276 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:48.002422 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33375436-90f6-400c-ba6c-cba7dc478276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:48.027275 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-33375436-90f6-400c-ba6c-cba7dc478276 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:48.028889 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-33375436-90f6-400c-ba6c-cba7dc478276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:48.128902 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1053/2094] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:48.136143 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10063010-488d-4b75-b298-49da558b6183 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:48.137173 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10063010-488d-4b75-b298-49da558b6183 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:48.137672 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10063010-488d-4b75-b298-49da558b6183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:48.165664 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-10063010-488d-4b75-b298-49da558b6183 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:48.167449 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-10063010-488d-4b75-b298-49da558b6183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:48.176320 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1042/2095] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:48.181150 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33375436-90f6-400c-ba6c-cba7dc478276 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:48.193663 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33375436-90f6-400c-ba6c-cba7dc478276 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:48.200496 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3011d58e-09d1-441c-9ad2-b271e10ac028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:48.201586 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3011d58e-09d1-441c-9ad2-b271e10ac028 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:48.201970 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3011d58e-09d1-441c-9ad2-b271e10ac028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:48.202199 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3011d58e-09d1-441c-9ad2-b271e10ac028 admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:48.202943 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3011d58e-09d1-441c-9ad2-b271e10ac028 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:17:48.203304 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3011d58e-09d1-441c-9ad2-b271e10ac028 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:48.205843 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3011d58e-09d1-441c-9ad2-b271e10ac028 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:48.216364 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:48.233991 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1054/2096] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:17:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:17:48.234882 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10063010-488d-4b75-b298-49da558b6183 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:48.245231 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10063010-488d-4b75-b298-49da558b6183 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:48.250712 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147a328d-6b26-4146-8f87-d5da5cad4360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:48.251400 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a328d-6b26-4146-8f87-d5da5cad4360 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:48.251894 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a328d-6b26-4146-8f87-d5da5cad4360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:48.252169 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a328d-6b26-4146-8f87-d5da5cad4360 admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:48.253057 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147a328d-6b26-4146-8f87-d5da5cad4360 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:48.255286 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147a328d-6b26-4146-8f87-d5da5cad4360 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:48.293786 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1043/2097] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:17:48 2022] POST /identity/v3/projects => generated 381 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:17:48.298058 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3011d58e-09d1-441c-9ad2-b271e10ac028 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:48.311537 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3011d58e-09d1-441c-9ad2-b271e10ac028 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:48.388596 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241fabab-499f-4774-9799-aa2855156ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:48.389324 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241fabab-499f-4774-9799-aa2855156ab1 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:48.389610 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241fabab-499f-4774-9799-aa2855156ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:48.389900 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241fabab-499f-4774-9799-aa2855156ab1 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:48.390622 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241fabab-499f-4774-9799-aa2855156ab1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:48.392596 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241fabab-499f-4774-9799-aa2855156ab1 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:48.417492 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1055/2098] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:17:48 2022] POST /identity/v3/users => generated 291 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:48.422203 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147a328d-6b26-4146-8f87-d5da5cad4360 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:48.434510 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147a328d-6b26-4146-8f87-d5da5cad4360 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:48.439640 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982686b-ddff-43db-8169-3d847cae98e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:48.440558 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982686b-ddff-43db-8169-3d847cae98e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:48.440877 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982686b-ddff-43db-8169-3d847cae98e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:48.441291 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982686b-ddff-43db-8169-3d847cae98e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:48.442308 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3982686b-ddff-43db-8169-3d847cae98e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:48.444057 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3982686b-ddff-43db-8169-3d847cae98e0 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:48.458805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1044/2099] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:48.463607 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241fabab-499f-4774-9799-aa2855156ab1 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:48.502245 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241fabab-499f-4774-9799-aa2855156ab1 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:48.507611 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d1a11-b9f3-43a7-80f6-6290c50ef808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:48.508624 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d1a11-b9f3-43a7-80f6-6290c50ef808 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:48.509079 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d1a11-b9f3-43a7-80f6-6290c50ef808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:48.509411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d1a11-b9f3-43a7-80f6-6290c50ef808 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:48.510179 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891d1a11-b9f3-43a7-80f6-6290c50ef808 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:48.511943 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891d1a11-b9f3-43a7-80f6-6290c50ef808 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:48.529630 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1056/2100] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:48 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:48.531360 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982686b-ddff-43db-8169-3d847cae98e0 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:48.543577 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982686b-ddff-43db-8169-3d847cae98e0 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:48.549367 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a9b7bc-5df7-4f4c-832f-30fc6b6f486c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:48.550442 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a9b7bc-5df7-4f4c-832f-30fc6b6f486c admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:48.550796 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a9b7bc-5df7-4f4c-832f-30fc6b6f486c admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:48.551151 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a9b7bc-5df7-4f4c-832f-30fc6b6f486c admin admin] PATH_INFO: `/v3/projects/47f03f05833440cf8fa8368120babcec/users/fc412cdf221e468ebc981db8086ad5f4/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:48.567482 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a9b7bc-5df7-4f4c-832f-30fc6b6f486c admin admin] RBAC: Authorizing `identity:create_grant(project_id=47f03f05833440cf8fa8368120babcec, user_id=fc412cdf221e468ebc981db8086ad5f4, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:48.570504 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a9b7bc-5df7-4f4c-832f-30fc6b6f486c admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:48.591403 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1045/2101] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:17:48 2022] PUT /identity/v3/projects/47f03f05833440cf8fa8368120babcec/users/fc412cdf221e468ebc981db8086ad5f4/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:48.604021 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd6c9d-3a1d-4bae-a7df-9ebd6719d94b None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:48.604309 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd6c9d-3a1d-4bae-a7df-9ebd6719d94b None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:48.604582 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd6c9d-3a1d-4bae-a7df-9ebd6719d94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:48.642049 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-cafd6c9d-3a1d-4bae-a7df-9ebd6719d94b None None] MFA Rules not processed for user `fc412cdf221e468ebc981db8086ad5f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:48.643293 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-cafd6c9d-3a1d-4bae-a7df-9ebd6719d94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:48.724583 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1057/2102] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:48.729652 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a9b7bc-5df7-4f4c-832f-30fc6b6f486c admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:48.810849 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a9b7bc-5df7-4f4c-832f-30fc6b6f486c admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:48.818116 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0e798d-064f-4b9e-b374-397e6d6159b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:48.820247 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e798d-064f-4b9e-b374-397e6d6159b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:48.820891 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e798d-064f-4b9e-b374-397e6d6159b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:48.821244 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e798d-064f-4b9e-b374-397e6d6159b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:48.822217 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0e798d-064f-4b9e-b374-397e6d6159b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:48.824474 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0e798d-064f-4b9e-b374-397e6d6159b5 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:48.860172 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1046/2103] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:17:48 2022] POST /identity/v3/projects => generated 379 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:48.861346 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd6c9d-3a1d-4bae-a7df-9ebd6719d94b None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:48.875071 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd6c9d-3a1d-4bae-a7df-9ebd6719d94b None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:48.946534 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf1bd7-1f71-4f09-84cb-25746e82d33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:48.947240 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf1bd7-1f71-4f09-84cb-25746e82d33e admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:48.947519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf1bd7-1f71-4f09-84cb-25746e82d33e admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:48.947796 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf1bd7-1f71-4f09-84cb-25746e82d33e admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:48.948555 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcf1bd7-1f71-4f09-84cb-25746e82d33e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:48.950529 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcf1bd7-1f71-4f09-84cb-25746e82d33e admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:48.959782 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:48.977050 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1058/2104] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:17:48 2022] POST /identity/v3/users => generated 296 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:48.979984 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0e798d-064f-4b9e-b374-397e6d6159b5 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:48.990986 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0e798d-064f-4b9e-b374-397e6d6159b5 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:48.996893 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40aa230-53ba-4fdf-95f2-822bd5bf948e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:48.997660 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40aa230-53ba-4fdf-95f2-822bd5bf948e admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:48.997944 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40aa230-53ba-4fdf-95f2-822bd5bf948e admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:48.998224 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40aa230-53ba-4fdf-95f2-822bd5bf948e admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:48.998973 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40aa230-53ba-4fdf-95f2-822bd5bf948e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:49.000609 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40aa230-53ba-4fdf-95f2-822bd5bf948e admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:49.014699 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1047/2105] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:49.018507 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf1bd7-1f71-4f09-84cb-25746e82d33e admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:49.031686 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf1bd7-1f71-4f09-84cb-25746e82d33e admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:49.037401 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a1444-5b15-44b9-9aa6-19babec818aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:49.038427 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a1444-5b15-44b9-9aa6-19babec818aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:49.038769 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a1444-5b15-44b9-9aa6-19babec818aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:49.039168 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a1444-5b15-44b9-9aa6-19babec818aa admin admin] PATH_INFO: `/v3/projects/fcdc0ab3c4c54e54baa6472418189e95/users/1638d95838ec49dfb05d37466040e68e/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:49.058182 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425a1444-5b15-44b9-9aa6-19babec818aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcdc0ab3c4c54e54baa6472418189e95, user_id=1638d95838ec49dfb05d37466040e68e, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:49.060909 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425a1444-5b15-44b9-9aa6-19babec818aa admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:49.079168 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1059/2106] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:17:49 2022] PUT /identity/v3/projects/fcdc0ab3c4c54e54baa6472418189e95/users/1638d95838ec49dfb05d37466040e68e/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:49.083588 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40aa230-53ba-4fdf-95f2-822bd5bf948e admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:49.095045 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40aa230-53ba-4fdf-95f2-822bd5bf948e admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:49.125604 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:49.162015 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e8fd8-2e1f-431a-9e5c-72210116f3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:49.162830 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e8fd8-2e1f-431a-9e5c-72210116f3c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:49.163161 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e8fd8-2e1f-431a-9e5c-72210116f3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:49.163441 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e8fd8-2e1f-431a-9e5c-72210116f3c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:49.164399 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e8fd8-2e1f-431a-9e5c-72210116f3c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:49.166216 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e8fd8-2e1f-431a-9e5c-72210116f3c4 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:49.178844 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1048/2107] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:49 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:49.183721 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a1444-5b15-44b9-9aa6-19babec818aa admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:49.194551 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a1444-5b15-44b9-9aa6-19babec818aa admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:49.200374 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a8379-f72f-429d-9c23-8fd44254a223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:49.201391 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a8379-f72f-429d-9c23-8fd44254a223 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:49.201669 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a8379-f72f-429d-9c23-8fd44254a223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:49.201946 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a8379-f72f-429d-9c23-8fd44254a223 admin admin] PATH_INFO: `/v3/projects/fcdc0ab3c4c54e54baa6472418189e95/users/1638d95838ec49dfb05d37466040e68e/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:49.204174 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40a8379-f72f-429d-9c23-8fd44254a223 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcdc0ab3c4c54e54baa6472418189e95, user_id=1638d95838ec49dfb05d37466040e68e, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:49.207665 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40a8379-f72f-429d-9c23-8fd44254a223 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:49.221196 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1060/2108] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:17:49 2022] PUT /identity/v3/projects/fcdc0ab3c4c54e54baa6472418189e95/users/1638d95838ec49dfb05d37466040e68e/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:49.229792 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9f920-a58e-4cb4-86e5-1e803ddb7a70 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:49.229792 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9f920-a58e-4cb4-86e5-1e803ddb7a70 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:49.229792 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd9f920-a58e-4cb4-86e5-1e803ddb7a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:49.231773 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c274bd0b-f28d-427f-b203-e7c3d79d0a85 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:49.232124 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c274bd0b-f28d-427f-b203-e7c3d79d0a85 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:49.232460 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c274bd0b-f28d-427f-b203-e7c3d79d0a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:49.246390 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-cbd9f920-a58e-4cb4-86e5-1e803ddb7a70 None None] MFA Rules not processed for user `4e8428365f1148a2934dd4596deeb43a`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:49.247623 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-cbd9f920-a58e-4cb4-86e5-1e803ddb7a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:49.263142 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-c274bd0b-f28d-427f-b203-e7c3d79d0a85 None None] MFA Rules not processed for user `1638d95838ec49dfb05d37466040e68e`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:49.267938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-c274bd0b-f28d-427f-b203-e7c3d79d0a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:49.326825 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1049/2109] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:17:49.368999 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1061/2110] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:49 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:49.379988 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd9f920-a58e-4cb4-86e5-1e803ddb7a70 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:49.381718 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-cbd9f920-a58e-4cb4-86e5-1e803ddb7a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:49.467578 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd9f920-a58e-4cb4-86e5-1e803ddb7a70 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:49.472121 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01fec32-118a-42b6-ad70-3602ddb21ffe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:49.472800 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01fec32-118a-42b6-ad70-3602ddb21ffe service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:49.473116 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01fec32-118a-42b6-ad70-3602ddb21ffe service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:49.473393 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01fec32-118a-42b6-ad70-3602ddb21ffe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:49.488670 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01fec32-118a-42b6-ad70-3602ddb21ffe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:49.490207 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01fec32-118a-42b6-ad70-3602ddb21ffe service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:49.504052 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1050/2111] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:49 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:50.038020 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c274bd0b-f28d-427f-b203-e7c3d79d0a85 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:50.053707 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c274bd0b-f28d-427f-b203-e7c3d79d0a85 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:50.059638 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762eede-76bc-4bc9-b6c0-c107dea0d35a tempest-VolumeQuotaClassesTest-973310359 tempest-VolumeQuotaClassesTest-973310359-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1638d95838ec49dfb05d37466040e68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fcdc0ab3c4c54e54baa6472418189e95', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:50.061790 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762eede-76bc-4bc9-b6c0-c107dea0d35a tempest-VolumeQuotaClassesTest-973310359 tempest-VolumeQuotaClassesTest-973310359-project-admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:50.062090 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762eede-76bc-4bc9-b6c0-c107dea0d35a tempest-VolumeQuotaClassesTest-973310359 tempest-VolumeQuotaClassesTest-973310359-project-admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:50.062374 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762eede-76bc-4bc9-b6c0-c107dea0d35a tempest-VolumeQuotaClassesTest-973310359 tempest-VolumeQuotaClassesTest-973310359-project-admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:50.063135 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762eede-76bc-4bc9-b6c0-c107dea0d35a tempest-VolumeQuotaClassesTest-973310359 tempest-VolumeQuotaClassesTest-973310359-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:17:50.063447 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762eede-76bc-4bc9-b6c0-c107dea0d35a tempest-VolumeQuotaClassesTest-973310359 tempest-VolumeQuotaClassesTest-973310359-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:50.065377 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762eede-76bc-4bc9-b6c0-c107dea0d35a tempest-VolumeQuotaClassesTest-973310359 tempest-VolumeQuotaClassesTest-973310359-project-admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:50.084257 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1062/2112] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:17:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:17:50.090079 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01fec32-118a-42b6-ad70-3602ddb21ffe service cinder] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:50.105625 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01fec32-118a-42b6-ad70-3602ddb21ffe service cinder] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:50.110575 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b2bec-30ca-46f9-ba0a-0b4bcc4102f3 tempest-VolumeQuotaClassesTest-973310359 tempest-VolumeQuotaClassesTest-973310359-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1638d95838ec49dfb05d37466040e68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fcdc0ab3c4c54e54baa6472418189e95', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:50.111251 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b2bec-30ca-46f9-ba0a-0b4bcc4102f3 tempest-VolumeQuotaClassesTest-973310359 tempest-VolumeQuotaClassesTest-973310359-project-admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:50.111537 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b2bec-30ca-46f9-ba0a-0b4bcc4102f3 tempest-VolumeQuotaClassesTest-973310359 tempest-VolumeQuotaClassesTest-973310359-project-admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:50.111830 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b2bec-30ca-46f9-ba0a-0b4bcc4102f3 tempest-VolumeQuotaClassesTest-973310359 tempest-VolumeQuotaClassesTest-973310359-project-admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:50.112558 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8b2bec-30ca-46f9-ba0a-0b4bcc4102f3 tempest-VolumeQuotaClassesTest-973310359 tempest-VolumeQuotaClassesTest-973310359-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:50.114563 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8b2bec-30ca-46f9-ba0a-0b4bcc4102f3 tempest-VolumeQuotaClassesTest-973310359 tempest-VolumeQuotaClassesTest-973310359-project-admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:50.155007 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1051/2113] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:17:50 2022] POST /identity/v3/projects => generated 355 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:50.159224 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762eede-76bc-4bc9-b6c0-c107dea0d35a tempest-VolumeQuotaClassesTest-973310359 tempest-VolumeQuotaClassesTest-973310359-project-admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:50.171167 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762eede-76bc-4bc9-b6c0-c107dea0d35a tempest-VolumeQuotaClassesTest-973310359 tempest-VolumeQuotaClassesTest-973310359-project-admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:50.255054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29891498-b609-4c4c-b789-8264f1288389 tempest-VolumeQuotaClassesTest-973310359 tempest-VolumeQuotaClassesTest-973310359-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1638d95838ec49dfb05d37466040e68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fcdc0ab3c4c54e54baa6472418189e95', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:50.255731 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29891498-b609-4c4c-b789-8264f1288389 tempest-VolumeQuotaClassesTest-973310359 tempest-VolumeQuotaClassesTest-973310359-project-admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:50.256223 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29891498-b609-4c4c-b789-8264f1288389 tempest-VolumeQuotaClassesTest-973310359 tempest-VolumeQuotaClassesTest-973310359-project-admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:50.256655 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29891498-b609-4c4c-b789-8264f1288389 tempest-VolumeQuotaClassesTest-973310359 tempest-VolumeQuotaClassesTest-973310359-project-admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:50.257814 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29891498-b609-4c4c-b789-8264f1288389 tempest-VolumeQuotaClassesTest-973310359 tempest-VolumeQuotaClassesTest-973310359-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:17:50.258300 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29891498-b609-4c4c-b789-8264f1288389 tempest-VolumeQuotaClassesTest-973310359 tempest-VolumeQuotaClassesTest-973310359-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:50.260504 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29891498-b609-4c4c-b789-8264f1288389 tempest-VolumeQuotaClassesTest-973310359 tempest-VolumeQuotaClassesTest-973310359-project-admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:50.276017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1063/2114] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:17:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:17:50.321109 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b2bec-30ca-46f9-ba0a-0b4bcc4102f3 tempest-VolumeQuotaClassesTest-973310359 tempest-VolumeQuotaClassesTest-973310359-project-admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:50.334915 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b2bec-30ca-46f9-ba0a-0b4bcc4102f3 tempest-VolumeQuotaClassesTest-973310359 tempest-VolumeQuotaClassesTest-973310359-project-admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:50.340534 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f051a5e-af74-48a5-9c96-6b4dcace1459 tempest-VolumeQuotaClassesTest-973310359 tempest-VolumeQuotaClassesTest-973310359-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1638d95838ec49dfb05d37466040e68e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fcdc0ab3c4c54e54baa6472418189e95', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:50.341550 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f051a5e-af74-48a5-9c96-6b4dcace1459 tempest-VolumeQuotaClassesTest-973310359 tempest-VolumeQuotaClassesTest-973310359-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:50.342026 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f051a5e-af74-48a5-9c96-6b4dcace1459 tempest-VolumeQuotaClassesTest-973310359 tempest-VolumeQuotaClassesTest-973310359-project-admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:50.342306 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f051a5e-af74-48a5-9c96-6b4dcace1459 tempest-VolumeQuotaClassesTest-973310359 tempest-VolumeQuotaClassesTest-973310359-project-admin] PATH_INFO: `/v3/projects/83fb8b80e29144f1979c334d3df3bc59` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:50.343541 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f051a5e-af74-48a5-9c96-6b4dcace1459 tempest-VolumeQuotaClassesTest-973310359 tempest-VolumeQuotaClassesTest-973310359-project-admin] RBAC: Authorizing `identity:delete_project(project_id=83fb8b80e29144f1979c334d3df3bc59)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:50.345503 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f051a5e-af74-48a5-9c96-6b4dcace1459 tempest-VolumeQuotaClassesTest-973310359 tempest-VolumeQuotaClassesTest-973310359-project-admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:50.409305 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:50.415666 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-4f051a5e-af74-48a5-9c96-6b4dcace1459 tempest-VolumeQuotaClassesTest-973310359 tempest-VolumeQuotaClassesTest-973310359-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83fb8b80e29144f1979c334d3df3bc59', 'request_id': 'req-4f051a5e-af74-48a5-9c96-6b4dcace1459'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:50.425720 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-4f051a5e-af74-48a5-9c96-6b4dcace1459 tempest-VolumeQuotaClassesTest-973310359 tempest-VolumeQuotaClassesTest-973310359-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83fb8b80e29144f1979c334d3df3bc59', 'request_id': 'req-4f051a5e-af74-48a5-9c96-6b4dcace1459'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:50.435749 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-4f051a5e-af74-48a5-9c96-6b4dcace1459 tempest-VolumeQuotaClassesTest-973310359 tempest-VolumeQuotaClassesTest-973310359-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83fb8b80e29144f1979c334d3df3bc59', 'request_id': 'req-4f051a5e-af74-48a5-9c96-6b4dcace1459'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:50.453302 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-4f051a5e-af74-48a5-9c96-6b4dcace1459 tempest-VolumeQuotaClassesTest-973310359 tempest-VolumeQuotaClassesTest-973310359-project-admin] The token cache is being invalidate because project 83fb8b80e29144f1979c334d3df3bc59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:17:50.453592 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-4f051a5e-af74-48a5-9c96-6b4dcace1459 tempest-VolumeQuotaClassesTest-973310359 tempest-VolumeQuotaClassesTest-973310359-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:50.454317 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-4f051a5e-af74-48a5-9c96-6b4dcace1459 tempest-VolumeQuotaClassesTest-973310359 tempest-VolumeQuotaClassesTest-973310359-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:50.456183 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1052/2115] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:17:50 2022] DELETE /identity/v3/projects/83fb8b80e29144f1979c334d3df3bc59 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:50.755105 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29891498-b609-4c4c-b789-8264f1288389 tempest-VolumeQuotaClassesTest-973310359 tempest-VolumeQuotaClassesTest-973310359-project-admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:50.759106 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-29891498-b609-4c4c-b789-8264f1288389 tempest-VolumeQuotaClassesTest-973310359 tempest-VolumeQuotaClassesTest-973310359-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:50.793657 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:50.836509 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29891498-b609-4c4c-b789-8264f1288389 tempest-VolumeQuotaClassesTest-973310359 tempest-VolumeQuotaClassesTest-973310359-project-admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:50.897169 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d400bd-0007-4c96-b682-d7aafc5b5546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:50.898053 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d400bd-0007-4c96-b682-d7aafc5b5546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:50.898434 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d400bd-0007-4c96-b682-d7aafc5b5546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:50.899165 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d400bd-0007-4c96-b682-d7aafc5b5546 admin admin] PATH_INFO: `/v3/users/fc412cdf221e468ebc981db8086ad5f4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:50.900194 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d400bd-0007-4c96-b682-d7aafc5b5546 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc412cdf221e468ebc981db8086ad5f4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:50.902195 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d400bd-0007-4c96-b682-d7aafc5b5546 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:50.952377 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-f5d400bd-0007-4c96-b682-d7aafc5b5546 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc412cdf221e468ebc981db8086ad5f4'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:50.960377 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-f5d400bd-0007-4c96-b682-d7aafc5b5546 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc412cdf221e468ebc981db8086ad5f4'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:50.961143 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-f5d400bd-0007-4c96-b682-d7aafc5b5546 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc412cdf221e468ebc981db8086ad5f4'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:50.961745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-f5d400bd-0007-4c96-b682-d7aafc5b5546 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc412cdf221e468ebc981db8086ad5f4'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:50.970472 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-f5d400bd-0007-4c96-b682-d7aafc5b5546 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc412cdf221e468ebc981db8086ad5f4'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:50.992355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1064/2116] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:17:50 2022] DELETE /identity/v3/users/fc412cdf221e468ebc981db8086ad5f4 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:17:50.997281 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88bd628-282b-4d9e-b8bc-bcb131af8c50 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:50.997577 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88bd628-282b-4d9e-b8bc-bcb131af8c50 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:50.997864 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88bd628-282b-4d9e-b8bc-bcb131af8c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:51.034858 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-d88bd628-282b-4d9e-b8bc-bcb131af8c50 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:51.036046 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-d88bd628-282b-4d9e-b8bc-bcb131af8c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:51.100331 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1053/2117] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:51.111313 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d400bd-0007-4c96-b682-d7aafc5b5546 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:51.112766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-f5d400bd-0007-4c96-b682-d7aafc5b5546 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:51.172711 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:51.176744 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d400bd-0007-4c96-b682-d7aafc5b5546 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:51.181115 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fcb6dd-f7c1-4e5e-8b70-41ad543b88ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:51.181786 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fcb6dd-f7c1-4e5e-8b70-41ad543b88ac service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:51.182076 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fcb6dd-f7c1-4e5e-8b70-41ad543b88ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:51.182356 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fcb6dd-f7c1-4e5e-8b70-41ad543b88ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:51.193409 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fcb6dd-f7c1-4e5e-8b70-41ad543b88ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:51.194881 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fcb6dd-f7c1-4e5e-8b70-41ad543b88ac service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:51.208975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1065/2118] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:51.552754 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88bd628-282b-4d9e-b8bc-bcb131af8c50 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:51.555169 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-d88bd628-282b-4d9e-b8bc-bcb131af8c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:51.577712 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88bd628-282b-4d9e-b8bc-bcb131af8c50 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:51.582887 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480a50f-168a-45df-af2b-7f250d9f7303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:51.583785 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480a50f-168a-45df-af2b-7f250d9f7303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:51.584070 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480a50f-168a-45df-af2b-7f250d9f7303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:51.584347 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480a50f-168a-45df-af2b-7f250d9f7303 admin admin] PATH_INFO: `/v3/projects/47f03f05833440cf8fa8368120babcec` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:51.585590 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b480a50f-168a-45df-af2b-7f250d9f7303 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47f03f05833440cf8fa8368120babcec)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:51.589189 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b480a50f-168a-45df-af2b-7f250d9f7303 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:51.643497 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-b480a50f-168a-45df-af2b-7f250d9f7303 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47f03f05833440cf8fa8368120babcec', 'request_id': 'req-b480a50f-168a-45df-af2b-7f250d9f7303'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:51.650678 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-b480a50f-168a-45df-af2b-7f250d9f7303 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47f03f05833440cf8fa8368120babcec', 'request_id': 'req-b480a50f-168a-45df-af2b-7f250d9f7303'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:51.661364 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-b480a50f-168a-45df-af2b-7f250d9f7303 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47f03f05833440cf8fa8368120babcec', 'request_id': 'req-b480a50f-168a-45df-af2b-7f250d9f7303'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:51.673807 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-b480a50f-168a-45df-af2b-7f250d9f7303 admin admin] The token cache is being invalidate because project 47f03f05833440cf8fa8368120babcec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:17:51.674098 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-b480a50f-168a-45df-af2b-7f250d9f7303 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:51.674895 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-b480a50f-168a-45df-af2b-7f250d9f7303 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:51.676642 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1054/2119] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:17:51 2022] DELETE /identity/v3/projects/47f03f05833440cf8fa8368120babcec => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:51.681371 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fcb6dd-f7c1-4e5e-8b70-41ad543b88ac service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:51.691111 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fcb6dd-f7c1-4e5e-8b70-41ad543b88ac service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:51.756490 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a2a5e-f7bb-42f5-bdbb-4db9f05cb045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:51.757424 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a2a5e-f7bb-42f5-bdbb-4db9f05cb045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:51.758009 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a2a5e-f7bb-42f5-bdbb-4db9f05cb045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:51.759000 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a2a5e-f7bb-42f5-bdbb-4db9f05cb045 admin admin] PATH_INFO: `/v3/users/1638d95838ec49dfb05d37466040e68e` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:51.760587 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5a2a5e-f7bb-42f5-bdbb-4db9f05cb045 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1638d95838ec49dfb05d37466040e68e)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:51.762821 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5a2a5e-f7bb-42f5-bdbb-4db9f05cb045 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:51.832396 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-7c5a2a5e-f7bb-42f5-bdbb-4db9f05cb045 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1638d95838ec49dfb05d37466040e68e'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:51.841246 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-7c5a2a5e-f7bb-42f5-bdbb-4db9f05cb045 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1638d95838ec49dfb05d37466040e68e'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:51.843529 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-7c5a2a5e-f7bb-42f5-bdbb-4db9f05cb045 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1638d95838ec49dfb05d37466040e68e'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:51.844140 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-7c5a2a5e-f7bb-42f5-bdbb-4db9f05cb045 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1638d95838ec49dfb05d37466040e68e'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:51.855199 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-7c5a2a5e-f7bb-42f5-bdbb-4db9f05cb045 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1638d95838ec49dfb05d37466040e68e'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:51.900209 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1066/2120] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:17:51 2022] DELETE /identity/v3/users/1638d95838ec49dfb05d37466040e68e => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:52.280432 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480a50f-168a-45df-af2b-7f250d9f7303 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:52.281983 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-b480a50f-168a-45df-af2b-7f250d9f7303 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:52.307394 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480a50f-168a-45df-af2b-7f250d9f7303 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:52.393444 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580966bb-bd32-4054-a045-773e1c115eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:52.394352 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580966bb-bd32-4054-a045-773e1c115eb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:52.394636 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580966bb-bd32-4054-a045-773e1c115eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:52.394907 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580966bb-bd32-4054-a045-773e1c115eb4 admin admin] PATH_INFO: `/v3/projects/fcdc0ab3c4c54e54baa6472418189e95` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:52.396132 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580966bb-bd32-4054-a045-773e1c115eb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcdc0ab3c4c54e54baa6472418189e95)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:52.398102 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580966bb-bd32-4054-a045-773e1c115eb4 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:52.400153 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:52.490758 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-580966bb-bd32-4054-a045-773e1c115eb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcdc0ab3c4c54e54baa6472418189e95', 'request_id': 'req-580966bb-bd32-4054-a045-773e1c115eb4'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:52.502643 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-580966bb-bd32-4054-a045-773e1c115eb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcdc0ab3c4c54e54baa6472418189e95', 'request_id': 'req-580966bb-bd32-4054-a045-773e1c115eb4'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:52.523585 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-580966bb-bd32-4054-a045-773e1c115eb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcdc0ab3c4c54e54baa6472418189e95', 'request_id': 'req-580966bb-bd32-4054-a045-773e1c115eb4'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:52.538434 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-580966bb-bd32-4054-a045-773e1c115eb4 admin admin] The token cache is being invalidate because project fcdc0ab3c4c54e54baa6472418189e95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:17:52.538654 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-580966bb-bd32-4054-a045-773e1c115eb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:52.539514 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-580966bb-bd32-4054-a045-773e1c115eb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:52.545471 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1055/2121] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:17:52 2022] DELETE /identity/v3/projects/fcdc0ab3c4c54e54baa6472418189e95 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:17:52.551591 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc9f0a-4051-4e66-8cc0-79a8889bbfd6 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:52.552320 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc9f0a-4051-4e66-8cc0-79a8889bbfd6 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:52.552682 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc9f0a-4051-4e66-8cc0-79a8889bbfd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:52.583260 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-0ecc9f0a-4051-4e66-8cc0-79a8889bbfd6 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:52.585452 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-0ecc9f0a-4051-4e66-8cc0-79a8889bbfd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:52.660170 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1067/2122] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:17:52.668120 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567639de-908a-4dfb-8d59-a1051b766e98 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:52.668711 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567639de-908a-4dfb-8d59-a1051b766e98 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:52.668711 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567639de-908a-4dfb-8d59-a1051b766e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:52.690131 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-567639de-908a-4dfb-8d59-a1051b766e98 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:52.691560 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-567639de-908a-4dfb-8d59-a1051b766e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:52.706807 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1056/2123] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:17:52.713363 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc9f0a-4051-4e66-8cc0-79a8889bbfd6 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:52.725883 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc9f0a-4051-4e66-8cc0-79a8889bbfd6 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:52.733232 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431f6490-3d14-4b0a-aa73-2ab84e1ac1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:52.734882 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f6490-3d14-4b0a-aa73-2ab84e1ac1db admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:52.735175 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f6490-3d14-4b0a-aa73-2ab84e1ac1db admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:52.735457 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f6490-3d14-4b0a-aa73-2ab84e1ac1db admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:52.736397 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431f6490-3d14-4b0a-aa73-2ab84e1ac1db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:17:52.736718 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431f6490-3d14-4b0a-aa73-2ab84e1ac1db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:52.738393 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431f6490-3d14-4b0a-aa73-2ab84e1ac1db admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:52.747495 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:52.760472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567639de-908a-4dfb-8d59-a1051b766e98 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:52.760686 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1068/2124] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:17:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:17:52.771824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567639de-908a-4dfb-8d59-a1051b766e98 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:52.778049 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83405197-76a2-43b8-9473-55d4b4d0ae72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:52.779045 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83405197-76a2-43b8-9473-55d4b4d0ae72 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:52.779540 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83405197-76a2-43b8-9473-55d4b4d0ae72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:52.780006 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83405197-76a2-43b8-9473-55d4b4d0ae72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:52.781006 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83405197-76a2-43b8-9473-55d4b4d0ae72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:52.783413 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83405197-76a2-43b8-9473-55d4b4d0ae72 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:52.824234 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1057/2125] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:17:52 2022] POST /identity/v3/projects => generated 401 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:52.828456 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431f6490-3d14-4b0a-aa73-2ab84e1ac1db admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:52.841263 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431f6490-3d14-4b0a-aa73-2ab84e1ac1db admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:52.909589 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4703a34a-523e-4713-966f-fec40a46f897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:52.910175 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703a34a-523e-4713-966f-fec40a46f897 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:52.910661 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703a34a-523e-4713-966f-fec40a46f897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:52.910757 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703a34a-523e-4713-966f-fec40a46f897 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:52.911428 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4703a34a-523e-4713-966f-fec40a46f897 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:52.913645 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4703a34a-523e-4713-966f-fec40a46f897 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:52.938787 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1069/2126] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:17:52 2022] POST /identity/v3/users => generated 301 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:17:52.946090 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83405197-76a2-43b8-9473-55d4b4d0ae72 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:52.964387 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83405197-76a2-43b8-9473-55d4b4d0ae72 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:52.971141 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1589f9-fd6b-4f57-801c-b7b6db355b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:52.972106 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1589f9-fd6b-4f57-801c-b7b6db355b74 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:52.972572 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1589f9-fd6b-4f57-801c-b7b6db355b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:52.973134 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1589f9-fd6b-4f57-801c-b7b6db355b74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:52.974080 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1589f9-fd6b-4f57-801c-b7b6db355b74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:52.975909 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1589f9-fd6b-4f57-801c-b7b6db355b74 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:52.990205 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1058/2127] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:52.994486 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4703a34a-523e-4713-966f-fec40a46f897 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:53.006104 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4703a34a-523e-4713-966f-fec40a46f897 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:53.013322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac707b79-d2a6-4bab-aa0b-da396b300bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:53.018335 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac707b79-d2a6-4bab-aa0b-da396b300bd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:53.018863 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac707b79-d2a6-4bab-aa0b-da396b300bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:53.019340 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac707b79-d2a6-4bab-aa0b-da396b300bd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:53.024546 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac707b79-d2a6-4bab-aa0b-da396b300bd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:53.026479 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac707b79-d2a6-4bab-aa0b-da396b300bd8 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:53.041423 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1070/2128] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:53.045184 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1589f9-fd6b-4f57-801c-b7b6db355b74 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:53.061060 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1589f9-fd6b-4f57-801c-b7b6db355b74 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:53.066592 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a6a4a0-7eb3-4030-b1ea-5870df65ea93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:53.071607 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a6a4a0-7eb3-4030-b1ea-5870df65ea93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:53.071971 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a6a4a0-7eb3-4030-b1ea-5870df65ea93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:53.072399 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a6a4a0-7eb3-4030-b1ea-5870df65ea93 admin admin] PATH_INFO: `/v3/projects/256d31ec7792424bae2b8934a919a54b/users/9b045cded7304828909773f5e3d54801/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:53.089380 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a6a4a0-7eb3-4030-b1ea-5870df65ea93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=256d31ec7792424bae2b8934a919a54b, user_id=9b045cded7304828909773f5e3d54801, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:53.092073 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a6a4a0-7eb3-4030-b1ea-5870df65ea93 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:53.121013 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1059/2129] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:17:53 2022] PUT /identity/v3/projects/256d31ec7792424bae2b8934a919a54b/users/9b045cded7304828909773f5e3d54801/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:53.130505 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13032dcf-a5c0-4b35-852b-da29495bbc5a None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:53.130952 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13032dcf-a5c0-4b35-852b-da29495bbc5a None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:53.131227 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13032dcf-a5c0-4b35-852b-da29495bbc5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:53.190506 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-13032dcf-a5c0-4b35-852b-da29495bbc5a None None] MFA Rules not processed for user `9b045cded7304828909773f5e3d54801`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:53.192072 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-13032dcf-a5c0-4b35-852b-da29495bbc5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:53.301069 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1071/2130] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:53 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:53.304655 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a6a4a0-7eb3-4030-b1ea-5870df65ea93 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:53.377376 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a6a4a0-7eb3-4030-b1ea-5870df65ea93 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:53.382222 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c2139-a8fd-4a42-bf03-d01e3f1c1346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:53.383281 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c2139-a8fd-4a42-bf03-d01e3f1c1346 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:53.383426 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c2139-a8fd-4a42-bf03-d01e3f1c1346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:53.383695 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c2139-a8fd-4a42-bf03-d01e3f1c1346 admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:53.384396 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c2139-a8fd-4a42-bf03-d01e3f1c1346 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:53.386355 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c2139-a8fd-4a42-bf03-d01e3f1c1346 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:53.417648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1060/2131] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:17:53 2022] POST /identity/v3/projects => generated 399 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:17:53.421986 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13032dcf-a5c0-4b35-852b-da29495bbc5a None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:53.434766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13032dcf-a5c0-4b35-852b-da29495bbc5a None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:53.504710 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1174e2db-f58c-4cd7-ad34-c1354ff33929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:53.505459 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1174e2db-f58c-4cd7-ad34-c1354ff33929 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:53.505738 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1174e2db-f58c-4cd7-ad34-c1354ff33929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:53.506110 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1174e2db-f58c-4cd7-ad34-c1354ff33929 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:53.506839 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1174e2db-f58c-4cd7-ad34-c1354ff33929 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:53.509300 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1174e2db-f58c-4cd7-ad34-c1354ff33929 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:53.521296 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:53.533992 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1072/2132] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:17:53 2022] POST /identity/v3/users => generated 306 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:53.538850 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c2139-a8fd-4a42-bf03-d01e3f1c1346 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:53.550986 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c2139-a8fd-4a42-bf03-d01e3f1c1346 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:53.556676 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b948fad9-c704-4f35-a259-cab172b14347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:53.557401 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b948fad9-c704-4f35-a259-cab172b14347 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:53.557684 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b948fad9-c704-4f35-a259-cab172b14347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:53.558408 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b948fad9-c704-4f35-a259-cab172b14347 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:53.558691 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b948fad9-c704-4f35-a259-cab172b14347 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:53.560303 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b948fad9-c704-4f35-a259-cab172b14347 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:53.566308 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:53.574202 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1061/2133] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:17:53.578330 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1174e2db-f58c-4cd7-ad34-c1354ff33929 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:53.589445 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1174e2db-f58c-4cd7-ad34-c1354ff33929 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:53.594458 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7afb57a-aaab-4c43-985c-db41edde39f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:53.595285 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7afb57a-aaab-4c43-985c-db41edde39f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:53.595573 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7afb57a-aaab-4c43-985c-db41edde39f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:53.595851 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7afb57a-aaab-4c43-985c-db41edde39f6 admin admin] PATH_INFO: `/v3/projects/860d3bbc1aa7497ea99e8b809f5f1e82/users/724d66ad43f3495f855c6cf267bec579/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:53.612158 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7afb57a-aaab-4c43-985c-db41edde39f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=860d3bbc1aa7497ea99e8b809f5f1e82, user_id=724d66ad43f3495f855c6cf267bec579, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:53.615464 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7afb57a-aaab-4c43-985c-db41edde39f6 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:53.631899 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1073/2134] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:17:53 2022] PUT /identity/v3/projects/860d3bbc1aa7497ea99e8b809f5f1e82/users/724d66ad43f3495f855c6cf267bec579/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:53.636043 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b948fad9-c704-4f35-a259-cab172b14347 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:53.647130 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b948fad9-c704-4f35-a259-cab172b14347 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:53.713855 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9efe36-91ca-405f-9bed-6a978cd60951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:53.714628 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9efe36-91ca-405f-9bed-6a978cd60951 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:53.714864 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9efe36-91ca-405f-9bed-6a978cd60951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:53.715153 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9efe36-91ca-405f-9bed-6a978cd60951 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:53.716018 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9efe36-91ca-405f-9bed-6a978cd60951 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:53.717812 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9efe36-91ca-405f-9bed-6a978cd60951 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:53.729625 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1062/2135] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:17:53.734755 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7afb57a-aaab-4c43-985c-db41edde39f6 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:53.747690 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7afb57a-aaab-4c43-985c-db41edde39f6 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:53.759847 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9342abf-e71b-43fb-a49d-a4710fd3c38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:53.760807 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9342abf-e71b-43fb-a49d-a4710fd3c38e admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:53.761123 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9342abf-e71b-43fb-a49d-a4710fd3c38e admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:53.761449 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9342abf-e71b-43fb-a49d-a4710fd3c38e admin admin] PATH_INFO: `/v3/projects/860d3bbc1aa7497ea99e8b809f5f1e82/users/724d66ad43f3495f855c6cf267bec579/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:53.763490 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9342abf-e71b-43fb-a49d-a4710fd3c38e admin admin] RBAC: Authorizing `identity:create_grant(project_id=860d3bbc1aa7497ea99e8b809f5f1e82, user_id=724d66ad43f3495f855c6cf267bec579, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:53.766279 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9342abf-e71b-43fb-a49d-a4710fd3c38e admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:53.782791 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1074/2136] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:17:53 2022] PUT /identity/v3/projects/860d3bbc1aa7497ea99e8b809f5f1e82/users/724d66ad43f3495f855c6cf267bec579/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:53.794083 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf0ff8-f0c0-4605-b233-e984754aa20f None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:53.794370 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf0ff8-f0c0-4605-b233-e984754aa20f None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:53.794650 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcf0ff8-f0c0-4605-b233-e984754aa20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:53.829830 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-3fcf0ff8-f0c0-4605-b233-e984754aa20f None None] MFA Rules not processed for user `724d66ad43f3495f855c6cf267bec579`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:53.831094 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-3fcf0ff8-f0c0-4605-b233-e984754aa20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:53.909763 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1063/2137] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:53 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:53.921902 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9342abf-e71b-43fb-a49d-a4710fd3c38e admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:53.924075 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-c9342abf-e71b-43fb-a49d-a4710fd3c38e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:53.988905 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9342abf-e71b-43fb-a49d-a4710fd3c38e admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:53.993566 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36474803-c770-47df-8b5b-7af75a37dcd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:53.994250 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36474803-c770-47df-8b5b-7af75a37dcd7 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:53.994538 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36474803-c770-47df-8b5b-7af75a37dcd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:53.994811 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36474803-c770-47df-8b5b-7af75a37dcd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:54.006469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36474803-c770-47df-8b5b-7af75a37dcd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:54.008231 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36474803-c770-47df-8b5b-7af75a37dcd7 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:54.020079 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1075/2138] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:53 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:54.106952 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcf0ff8-f0c0-4605-b233-e984754aa20f None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:54.117485 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcf0ff8-f0c0-4605-b233-e984754aa20f None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:54.122797 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638f5ce1-c116-4eba-a718-bedb1ea97353 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:54.123471 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f5ce1-c116-4eba-a718-bedb1ea97353 service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:54.123755 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f5ce1-c116-4eba-a718-bedb1ea97353 service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:54.124029 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638f5ce1-c116-4eba-a718-bedb1ea97353 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:54.191392 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638f5ce1-c116-4eba-a718-bedb1ea97353 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:54.193210 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638f5ce1-c116-4eba-a718-bedb1ea97353 service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:54.209473 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1064/2139] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:54 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:55.215725 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36474803-c770-47df-8b5b-7af75a37dcd7 service cinder] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:55.217209 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-36474803-c770-47df-8b5b-7af75a37dcd7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:55.309273 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36474803-c770-47df-8b5b-7af75a37dcd7 service cinder] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:55.314378 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4867d-1832-4122-bdac-071240eab78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:55.315470 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4867d-1832-4122-bdac-071240eab78f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:55.315832 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4867d-1832-4122-bdac-071240eab78f admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:55.316151 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4867d-1832-4122-bdac-071240eab78f admin admin] PATH_INFO: `/v3/users/975c6ffce82e4c5db815695dfe3c235b` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:55.317357 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b4867d-1832-4122-bdac-071240eab78f admin admin] RBAC: Authorizing `identity:delete_user(user_id=975c6ffce82e4c5db815695dfe3c235b)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:55.328125 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b4867d-1832-4122-bdac-071240eab78f admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:55.378420 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-83b4867d-1832-4122-bdac-071240eab78f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '975c6ffce82e4c5db815695dfe3c235b'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:55.385295 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-83b4867d-1832-4122-bdac-071240eab78f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '975c6ffce82e4c5db815695dfe3c235b'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:55.385958 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-83b4867d-1832-4122-bdac-071240eab78f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '975c6ffce82e4c5db815695dfe3c235b'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:55.386556 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-83b4867d-1832-4122-bdac-071240eab78f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '975c6ffce82e4c5db815695dfe3c235b'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:55.394715 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-83b4867d-1832-4122-bdac-071240eab78f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '975c6ffce82e4c5db815695dfe3c235b'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:55.395946 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:55.415419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1076/2140] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:17:55 2022] DELETE /identity/v3/users/975c6ffce82e4c5db815695dfe3c235b => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:55.420022 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22228dd4-9784-41eb-af27-86940e381d89 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:55.420505 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22228dd4-9784-41eb-af27-86940e381d89 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:55.420615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22228dd4-9784-41eb-af27-86940e381d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:55.439206 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-22228dd4-9784-41eb-af27-86940e381d89 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:55.440467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-22228dd4-9784-41eb-af27-86940e381d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:55.455378 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:55.514232 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1065/2141] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:55.524279 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4867d-1832-4122-bdac-071240eab78f admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:55.525969 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-83b4867d-1832-4122-bdac-071240eab78f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:55.590485 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4867d-1832-4122-bdac-071240eab78f admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:55.594854 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04533f32-ad7f-4885-9e3a-2b7e39a3329f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:55.595599 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04533f32-ad7f-4885-9e3a-2b7e39a3329f service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:55.595952 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04533f32-ad7f-4885-9e3a-2b7e39a3329f service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:55.596296 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04533f32-ad7f-4885-9e3a-2b7e39a3329f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:55.607712 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04533f32-ad7f-4885-9e3a-2b7e39a3329f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:55.609970 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04533f32-ad7f-4885-9e3a-2b7e39a3329f service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:55.624969 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1077/2142] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:55.951770 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22228dd4-9784-41eb-af27-86940e381d89 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:55.953430 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-22228dd4-9784-41eb-af27-86940e381d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:55.975182 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22228dd4-9784-41eb-af27-86940e381d89 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:55.980306 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bb95fc-8b03-4e3e-9fab-030f67d3d708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:55.982396 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb95fc-8b03-4e3e-9fab-030f67d3d708 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:55.982747 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb95fc-8b03-4e3e-9fab-030f67d3d708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:55.983522 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb95fc-8b03-4e3e-9fab-030f67d3d708 admin admin] PATH_INFO: `/v3/projects/04bdba8864c8481180acedc32712fafe` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:55.985127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bb95fc-8b03-4e3e-9fab-030f67d3d708 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04bdba8864c8481180acedc32712fafe)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:55.987064 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bb95fc-8b03-4e3e-9fab-030f67d3d708 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:56.047511 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-36bb95fc-8b03-4e3e-9fab-030f67d3d708 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04bdba8864c8481180acedc32712fafe', 'request_id': 'req-36bb95fc-8b03-4e3e-9fab-030f67d3d708'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:56.053770 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-36bb95fc-8b03-4e3e-9fab-030f67d3d708 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04bdba8864c8481180acedc32712fafe', 'request_id': 'req-36bb95fc-8b03-4e3e-9fab-030f67d3d708'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:56.062534 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-36bb95fc-8b03-4e3e-9fab-030f67d3d708 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04bdba8864c8481180acedc32712fafe', 'request_id': 'req-36bb95fc-8b03-4e3e-9fab-030f67d3d708'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:56.075275 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-36bb95fc-8b03-4e3e-9fab-030f67d3d708 admin admin] The token cache is being invalidate because project 04bdba8864c8481180acedc32712fafe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:17:56.075643 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-36bb95fc-8b03-4e3e-9fab-030f67d3d708 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:56.076336 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-36bb95fc-8b03-4e3e-9fab-030f67d3d708 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:56.077961 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1066/2143] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:17:55 2022] DELETE /identity/v3/projects/04bdba8864c8481180acedc32712fafe => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:56.083019 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274a5ab-7ea0-47fb-a143-c4f08444eae8 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:56.083312 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274a5ab-7ea0-47fb-a143-c4f08444eae8 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:56.083592 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274a5ab-7ea0-47fb-a143-c4f08444eae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:56.102933 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-f274a5ab-7ea0-47fb-a143-c4f08444eae8 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:56.104479 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-f274a5ab-7ea0-47fb-a143-c4f08444eae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:56.207903 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1078/2144] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:56.217503 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caf462c-16cf-4fcd-a2c1-26e339ba4159 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:56.217939 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caf462c-16cf-4fcd-a2c1-26e339ba4159 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:56.218313 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caf462c-16cf-4fcd-a2c1-26e339ba4159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:56.246205 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-6caf462c-16cf-4fcd-a2c1-26e339ba4159 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:56.247708 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-6caf462c-16cf-4fcd-a2c1-26e339ba4159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:56.261186 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1067/2145] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:17:56.261551 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f274a5ab-7ea0-47fb-a143-c4f08444eae8 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:56.272168 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f274a5ab-7ea0-47fb-a143-c4f08444eae8 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:56.277972 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc907db2-953c-4d0a-bb35-fb727e4c8d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:56.279152 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc907db2-953c-4d0a-bb35-fb727e4c8d12 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:56.279517 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc907db2-953c-4d0a-bb35-fb727e4c8d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:56.279849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc907db2-953c-4d0a-bb35-fb727e4c8d12 admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:56.280845 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc907db2-953c-4d0a-bb35-fb727e4c8d12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:17:56.281305 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc907db2-953c-4d0a-bb35-fb727e4c8d12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:56.283206 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc907db2-953c-4d0a-bb35-fb727e4c8d12 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:56.296700 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1079/2146] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:17:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:17:56.302454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caf462c-16cf-4fcd-a2c1-26e339ba4159 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:56.314383 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caf462c-16cf-4fcd-a2c1-26e339ba4159 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:56.319376 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c104604-3cb2-4254-8dd6-0c721d2ae475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:56.320058 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c104604-3cb2-4254-8dd6-0c721d2ae475 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:56.320339 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c104604-3cb2-4254-8dd6-0c721d2ae475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:56.320647 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c104604-3cb2-4254-8dd6-0c721d2ae475 admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:56.321418 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c104604-3cb2-4254-8dd6-0c721d2ae475 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:56.323366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c104604-3cb2-4254-8dd6-0c721d2ae475 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:56.361353 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1068/2147] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:17:56 2022] POST /identity/v3/projects => generated 399 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:56.363153 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc907db2-953c-4d0a-bb35-fb727e4c8d12 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:56.377185 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc907db2-953c-4d0a-bb35-fb727e4c8d12 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:56.482612 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8231a3-5bcf-4a08-8d5f-9d2bd690aac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:56.483298 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8231a3-5bcf-4a08-8d5f-9d2bd690aac4 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:56.483606 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8231a3-5bcf-4a08-8d5f-9d2bd690aac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:56.483890 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8231a3-5bcf-4a08-8d5f-9d2bd690aac4 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:56.484575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8231a3-5bcf-4a08-8d5f-9d2bd690aac4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:56.489486 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8231a3-5bcf-4a08-8d5f-9d2bd690aac4 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:56.509557 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:56.520978 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1080/2148] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:17:56 2022] POST /identity/v3/users => generated 300 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:17:56.521617 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c104604-3cb2-4254-8dd6-0c721d2ae475 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:56.533110 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c104604-3cb2-4254-8dd6-0c721d2ae475 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:56.540063 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331e244-21d2-4333-a683-64eca5a6da71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:56.540810 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331e244-21d2-4333-a683-64eca5a6da71 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:56.541354 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331e244-21d2-4333-a683-64eca5a6da71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:56.541794 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331e244-21d2-4333-a683-64eca5a6da71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:56.542772 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331e244-21d2-4333-a683-64eca5a6da71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:56.545639 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331e244-21d2-4333-a683-64eca5a6da71 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:56.562762 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1069/2149] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:17:56.567843 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8231a3-5bcf-4a08-8d5f-9d2bd690aac4 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:56.582722 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8231a3-5bcf-4a08-8d5f-9d2bd690aac4 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:56.591264 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a988a37-bf4d-4a36-8bf4-7bca4ea0b5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:56.592450 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a988a37-bf4d-4a36-8bf4-7bca4ea0b5f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:56.593078 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a988a37-bf4d-4a36-8bf4-7bca4ea0b5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:56.593528 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a988a37-bf4d-4a36-8bf4-7bca4ea0b5f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:56.594897 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a988a37-bf4d-4a36-8bf4-7bca4ea0b5f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:56.597270 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a988a37-bf4d-4a36-8bf4-7bca4ea0b5f1 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:56.620084 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1081/2150] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:17:56.627827 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331e244-21d2-4333-a683-64eca5a6da71 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:56.644669 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331e244-21d2-4333-a683-64eca5a6da71 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:56.652573 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7028c0d6-5089-48bf-a5a2-a199f87e8213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:56.653820 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028c0d6-5089-48bf-a5a2-a199f87e8213 admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:56.654170 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028c0d6-5089-48bf-a5a2-a199f87e8213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:56.654599 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028c0d6-5089-48bf-a5a2-a199f87e8213 admin admin] PATH_INFO: `/v3/projects/a9fe138f9fd4454fbefdfc17fe392fe3/users/e0e1b99f79534c8a9aa10bb01b9aef9e/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:56.688045 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7028c0d6-5089-48bf-a5a2-a199f87e8213 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9fe138f9fd4454fbefdfc17fe392fe3, user_id=e0e1b99f79534c8a9aa10bb01b9aef9e, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:56.690490 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7028c0d6-5089-48bf-a5a2-a199f87e8213 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:56.706581 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1070/2151] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:17:56 2022] PUT /identity/v3/projects/a9fe138f9fd4454fbefdfc17fe392fe3/users/e0e1b99f79534c8a9aa10bb01b9aef9e/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:56.717585 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab736fe6-0eb5-4e25-bac1-ac5f4b5b63ba None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:56.717956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab736fe6-0eb5-4e25-bac1-ac5f4b5b63ba None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:56.718945 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab736fe6-0eb5-4e25-bac1-ac5f4b5b63ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:56.752276 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-ab736fe6-0eb5-4e25-bac1-ac5f4b5b63ba None None] MFA Rules not processed for user `e0e1b99f79534c8a9aa10bb01b9aef9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:56.753598 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ab736fe6-0eb5-4e25-bac1-ac5f4b5b63ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:56.833715 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1082/2152] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:56 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:56.846764 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7028c0d6-5089-48bf-a5a2-a199f87e8213 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:56.848588 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-7028c0d6-5089-48bf-a5a2-a199f87e8213 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:56.900428 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:56.917417 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7028c0d6-5089-48bf-a5a2-a199f87e8213 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:56.921480 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f4024-91f2-4e2c-9604-9901b1b846af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:56.922146 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f4024-91f2-4e2c-9604-9901b1b846af service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:56.922429 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f4024-91f2-4e2c-9604-9901b1b846af service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:56.922701 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f4024-91f2-4e2c-9604-9901b1b846af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:56.931845 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1f4024-91f2-4e2c-9604-9901b1b846af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:56.933326 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1f4024-91f2-4e2c-9604-9901b1b846af service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:56.942866 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1071/2153] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:56 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:17:58.602114 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab736fe6-0eb5-4e25-bac1-ac5f4b5b63ba None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:58.604436 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ab736fe6-0eb5-4e25-bac1-ac5f4b5b63ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:58.706781 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab736fe6-0eb5-4e25-bac1-ac5f4b5b63ba None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:58.714130 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7553d2c6-4e99-4309-96db-93006c9d0249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:58.715226 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553d2c6-4e99-4309-96db-93006c9d0249 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:58.715580 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553d2c6-4e99-4309-96db-93006c9d0249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:58.716024 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7553d2c6-4e99-4309-96db-93006c9d0249 admin admin] PATH_INFO: `/v3/users/9b045cded7304828909773f5e3d54801` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:58.717637 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7553d2c6-4e99-4309-96db-93006c9d0249 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b045cded7304828909773f5e3d54801)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:58.720496 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7553d2c6-4e99-4309-96db-93006c9d0249 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:58.809072 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-7553d2c6-4e99-4309-96db-93006c9d0249 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b045cded7304828909773f5e3d54801'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:58.830167 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-7553d2c6-4e99-4309-96db-93006c9d0249 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b045cded7304828909773f5e3d54801'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:58.831454 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-7553d2c6-4e99-4309-96db-93006c9d0249 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b045cded7304828909773f5e3d54801'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:58.832444 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-7553d2c6-4e99-4309-96db-93006c9d0249 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b045cded7304828909773f5e3d54801'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:58.854012 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-7553d2c6-4e99-4309-96db-93006c9d0249 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b045cded7304828909773f5e3d54801'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:58.882242 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:17:58.889160 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1083/2154] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:17:58 2022] DELETE /identity/v3/users/9b045cded7304828909773f5e3d54801 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:58.897509 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba6669-2978-46c1-8999-8da4cecd8d72 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:58.898187 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba6669-2978-46c1-8999-8da4cecd8d72 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:58.898521 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba6669-2978-46c1-8999-8da4cecd8d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:58.928320 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-c7ba6669-2978-46c1-8999-8da4cecd8d72 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:17:58.931035 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-c7ba6669-2978-46c1-8999-8da4cecd8d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:59.040520 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1072/2155] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:17:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:17:59.048124 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7553d2c6-4e99-4309-96db-93006c9d0249 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:59.051907 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-7553d2c6-4e99-4309-96db-93006c9d0249 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:59.127556 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7553d2c6-4e99-4309-96db-93006c9d0249 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:59.134928 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454454ce-fd00-45d4-8877-1747ccbfbebe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:59.135809 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454454ce-fd00-45d4-8877-1747ccbfbebe service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:59.136209 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454454ce-fd00-45d4-8877-1747ccbfbebe service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:59.136687 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454454ce-fd00-45d4-8877-1747ccbfbebe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:59.153455 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454454ce-fd00-45d4-8877-1747ccbfbebe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:59.154930 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454454ce-fd00-45d4-8877-1747ccbfbebe service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:59.167849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1084/2156] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:17:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:17:59.515461 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ba6669-2978-46c1-8999-8da4cecd8d72 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:59.517214 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-c7ba6669-2978-46c1-8999-8da4cecd8d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:17:59.533421 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ba6669-2978-46c1-8999-8da4cecd8d72 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:59.539615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca9dc5c-01ae-4b71-a4bc-6ea59acf30a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:59.540655 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9dc5c-01ae-4b71-a4bc-6ea59acf30a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:59.541039 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9dc5c-01ae-4b71-a4bc-6ea59acf30a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:59.541384 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9dc5c-01ae-4b71-a4bc-6ea59acf30a4 admin admin] PATH_INFO: `/v3/projects/256d31ec7792424bae2b8934a919a54b` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:59.542894 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca9dc5c-01ae-4b71-a4bc-6ea59acf30a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=256d31ec7792424bae2b8934a919a54b)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:59.545325 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca9dc5c-01ae-4b71-a4bc-6ea59acf30a4 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:59.629398 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-eca9dc5c-01ae-4b71-a4bc-6ea59acf30a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '256d31ec7792424bae2b8934a919a54b', 'request_id': 'req-eca9dc5c-01ae-4b71-a4bc-6ea59acf30a4'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:59.641147 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-eca9dc5c-01ae-4b71-a4bc-6ea59acf30a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '256d31ec7792424bae2b8934a919a54b', 'request_id': 'req-eca9dc5c-01ae-4b71-a4bc-6ea59acf30a4'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:59.659098 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-eca9dc5c-01ae-4b71-a4bc-6ea59acf30a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '256d31ec7792424bae2b8934a919a54b', 'request_id': 'req-eca9dc5c-01ae-4b71-a4bc-6ea59acf30a4'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:59.676204 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-eca9dc5c-01ae-4b71-a4bc-6ea59acf30a4 admin admin] The token cache is being invalidate because project 256d31ec7792424bae2b8934a919a54b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:17:59.676204 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-eca9dc5c-01ae-4b71-a4bc-6ea59acf30a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:59.677313 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-eca9dc5c-01ae-4b71-a4bc-6ea59acf30a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:59.680027 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1073/2157] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:17:59 2022] DELETE /identity/v3/projects/256d31ec7792424bae2b8934a919a54b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:17:59.686248 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454454ce-fd00-45d4-8877-1747ccbfbebe service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:17:59.701669 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454454ce-fd00-45d4-8877-1747ccbfbebe service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:17:59.808557 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bc8b63-110b-4733-b5c2-5d4ca5c59fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:17:59.810219 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc8b63-110b-4733-b5c2-5d4ca5c59fa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:17:59.810962 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc8b63-110b-4733-b5c2-5d4ca5c59fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:17:59.812418 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc8b63-110b-4733-b5c2-5d4ca5c59fa1 admin admin] PATH_INFO: `/v3/users/724d66ad43f3495f855c6cf267bec579` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:17:59.814966 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bc8b63-110b-4733-b5c2-5d4ca5c59fa1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=724d66ad43f3495f855c6cf267bec579)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:17:59.820423 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bc8b63-110b-4733-b5c2-5d4ca5c59fa1 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:17:59.922653 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-44bc8b63-110b-4733-b5c2-5d4ca5c59fa1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '724d66ad43f3495f855c6cf267bec579'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:59.930530 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-44bc8b63-110b-4733-b5c2-5d4ca5c59fa1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '724d66ad43f3495f855c6cf267bec579'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:59.931365 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-44bc8b63-110b-4733-b5c2-5d4ca5c59fa1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '724d66ad43f3495f855c6cf267bec579'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:59.932061 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-44bc8b63-110b-4733-b5c2-5d4ca5c59fa1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '724d66ad43f3495f855c6cf267bec579'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:59.942042 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-44bc8b63-110b-4733-b5c2-5d4ca5c59fa1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '724d66ad43f3495f855c6cf267bec579'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:17:59.973697 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1085/2158] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:17:59 2022] DELETE /identity/v3/users/724d66ad43f3495f855c6cf267bec579 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:18:00.422306 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca9dc5c-01ae-4b71-a4bc-6ea59acf30a4 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:00.423983 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-eca9dc5c-01ae-4b71-a4bc-6ea59acf30a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:00.439601 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca9dc5c-01ae-4b71-a4bc-6ea59acf30a4 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:00.511121 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce0f83-6d94-47c6-ba01-a6152bd612d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:00.512013 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce0f83-6d94-47c6-ba01-a6152bd612d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:00.512292 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce0f83-6d94-47c6-ba01-a6152bd612d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:00.512567 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce0f83-6d94-47c6-ba01-a6152bd612d5 admin admin] PATH_INFO: `/v3/projects/860d3bbc1aa7497ea99e8b809f5f1e82` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:00.513888 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ce0f83-6d94-47c6-ba01-a6152bd612d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=860d3bbc1aa7497ea99e8b809f5f1e82)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:00.515893 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ce0f83-6d94-47c6-ba01-a6152bd612d5 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:00.528366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:00.588717 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-64ce0f83-6d94-47c6-ba01-a6152bd612d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '860d3bbc1aa7497ea99e8b809f5f1e82', 'request_id': 'req-64ce0f83-6d94-47c6-ba01-a6152bd612d5'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:00.597486 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-64ce0f83-6d94-47c6-ba01-a6152bd612d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '860d3bbc1aa7497ea99e8b809f5f1e82', 'request_id': 'req-64ce0f83-6d94-47c6-ba01-a6152bd612d5'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:00.609792 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-64ce0f83-6d94-47c6-ba01-a6152bd612d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '860d3bbc1aa7497ea99e8b809f5f1e82', 'request_id': 'req-64ce0f83-6d94-47c6-ba01-a6152bd612d5'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:00.625692 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-64ce0f83-6d94-47c6-ba01-a6152bd612d5 admin admin] The token cache is being invalidate because project 860d3bbc1aa7497ea99e8b809f5f1e82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:18:00.626143 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-64ce0f83-6d94-47c6-ba01-a6152bd612d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:00.627064 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-64ce0f83-6d94-47c6-ba01-a6152bd612d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:00.629058 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1074/2159] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:18:00 2022] DELETE /identity/v3/projects/860d3bbc1aa7497ea99e8b809f5f1e82 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:18:00.635537 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbcd73c-ba42-43f4-b0b7-5fa4c59960a4 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:00.636117 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbcd73c-ba42-43f4-b0b7-5fa4c59960a4 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:00.636678 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbcd73c-ba42-43f4-b0b7-5fa4c59960a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:00.662900 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-1cbcd73c-ba42-43f4-b0b7-5fa4c59960a4 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:00.665081 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-1cbcd73c-ba42-43f4-b0b7-5fa4c59960a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:00.686864 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:00.740268 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1086/2160] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:18:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:00.749471 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13564b4-5efe-4108-99cf-fce794bb572e None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:00.749758 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13564b4-5efe-4108-99cf-fce794bb572e None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:00.750076 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13564b4-5efe-4108-99cf-fce794bb572e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:00.776737 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-a13564b4-5efe-4108-99cf-fce794bb572e None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:00.779766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-a13564b4-5efe-4108-99cf-fce794bb572e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:00.789778 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1075/2161] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:18:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:00.795113 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbcd73c-ba42-43f4-b0b7-5fa4c59960a4 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:00.807232 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbcd73c-ba42-43f4-b0b7-5fa4c59960a4 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:00.814237 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3491b9-7e5a-401b-b334-4b210b509700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:00.815373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3491b9-7e5a-401b-b334-4b210b509700 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:00.815716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3491b9-7e5a-401b-b334-4b210b509700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:00.816193 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3491b9-7e5a-401b-b334-4b210b509700 admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:00.817223 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3491b9-7e5a-401b-b334-4b210b509700 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:18:00.817544 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3491b9-7e5a-401b-b334-4b210b509700 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:00.819228 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3491b9-7e5a-401b-b334-4b210b509700 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:00.832974 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1087/2162] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:18:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:18:00.838999 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13564b4-5efe-4108-99cf-fce794bb572e None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:00.850488 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13564b4-5efe-4108-99cf-fce794bb572e None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:00.859481 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1840ab-ad3f-4851-a100-bfe082da1217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:00.860351 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1840ab-ad3f-4851-a100-bfe082da1217 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:00.860720 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1840ab-ad3f-4851-a100-bfe082da1217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:00.861162 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1840ab-ad3f-4851-a100-bfe082da1217 admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:00.862085 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1840ab-ad3f-4851-a100-bfe082da1217 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:00.864762 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1840ab-ad3f-4851-a100-bfe082da1217 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:00.904280 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1076/2163] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:18:00 2022] POST /identity/v3/projects => generated 383 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:18:00.905691 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3491b9-7e5a-401b-b334-4b210b509700 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:00.918149 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3491b9-7e5a-401b-b334-4b210b509700 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:00.992870 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f41f42f-23ad-4d85-beea-7eca44f27551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:00.993646 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41f42f-23ad-4d85-beea-7eca44f27551 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:00.994004 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41f42f-23ad-4d85-beea-7eca44f27551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:00.994305 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41f42f-23ad-4d85-beea-7eca44f27551 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:00.995051 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f41f42f-23ad-4d85-beea-7eca44f27551 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:00.997190 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f41f42f-23ad-4d85-beea-7eca44f27551 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:01.023065 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1088/2164] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:18:00 2022] POST /identity/v3/users => generated 292 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:18:01.028327 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1840ab-ad3f-4851-a100-bfe082da1217 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:01.040646 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1840ab-ad3f-4851-a100-bfe082da1217 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:01.045917 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a047e84-79b2-4622-9c3b-b48ac4e3a7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:01.046603 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a047e84-79b2-4622-9c3b-b48ac4e3a7cb admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:01.046886 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a047e84-79b2-4622-9c3b-b48ac4e3a7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:01.047155 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a047e84-79b2-4622-9c3b-b48ac4e3a7cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:01.047906 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a047e84-79b2-4622-9c3b-b48ac4e3a7cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:01.049608 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a047e84-79b2-4622-9c3b-b48ac4e3a7cb admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:01.062771 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1077/2165] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:18:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:18:01.067250 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f41f42f-23ad-4d85-beea-7eca44f27551 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:01.079456 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f41f42f-23ad-4d85-beea-7eca44f27551 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:01.084892 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c3f864-286f-4552-af71-97b413e7b6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:01.085642 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3f864-286f-4552-af71-97b413e7b6bf admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:01.085957 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3f864-286f-4552-af71-97b413e7b6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:01.086398 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3f864-286f-4552-af71-97b413e7b6bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:01.087553 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c3f864-286f-4552-af71-97b413e7b6bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:01.089365 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c3f864-286f-4552-af71-97b413e7b6bf admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:01.102233 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1089/2166] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:18:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:18:01.107325 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a047e84-79b2-4622-9c3b-b48ac4e3a7cb admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:01.119257 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a047e84-79b2-4622-9c3b-b48ac4e3a7cb admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:01.124346 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5c3c5-99a0-4713-8236-0c4c8925200d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:01.125307 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5c3c5-99a0-4713-8236-0c4c8925200d admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:01.125777 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5c3c5-99a0-4713-8236-0c4c8925200d admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:01.125940 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5c3c5-99a0-4713-8236-0c4c8925200d admin admin] PATH_INFO: `/v3/projects/fab981f8468442bc9ad854bf7930ec03/users/8a622e94732a429192c57963cb4e499b/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:01.143932 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f5c3c5-99a0-4713-8236-0c4c8925200d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fab981f8468442bc9ad854bf7930ec03, user_id=8a622e94732a429192c57963cb4e499b, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:01.146867 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f5c3c5-99a0-4713-8236-0c4c8925200d admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:01.166407 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1078/2167] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:18:01 2022] PUT /identity/v3/projects/fab981f8468442bc9ad854bf7930ec03/users/8a622e94732a429192c57963cb4e499b/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:18:01.178902 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f65e70-a20d-47ad-876e-4a47cf46325e None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:01.179177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f65e70-a20d-47ad-876e-4a47cf46325e None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:01.179380 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f65e70-a20d-47ad-876e-4a47cf46325e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:01.227131 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-68f65e70-a20d-47ad-876e-4a47cf46325e None None] MFA Rules not processed for user `8a622e94732a429192c57963cb4e499b`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:01.229113 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-68f65e70-a20d-47ad-876e-4a47cf46325e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:01.311123 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1090/2168] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:18:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:01.325821 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5c3c5-99a0-4713-8236-0c4c8925200d admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:01.327835 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-43f5c3c5-99a0-4713-8236-0c4c8925200d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:01.403578 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5c3c5-99a0-4713-8236-0c4c8925200d admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:01.408758 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd8ad59-d09f-4474-81e4-818fa6d690dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:01.409896 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8ad59-d09f-4474-81e4-818fa6d690dc service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:01.410367 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8ad59-d09f-4474-81e4-818fa6d690dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:01.410778 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8ad59-d09f-4474-81e4-818fa6d690dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:01.427735 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd8ad59-d09f-4474-81e4-818fa6d690dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:01.429344 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd8ad59-d09f-4474-81e4-818fa6d690dc service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:01.440277 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1079/2169] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:18:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:18:04.217451 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f65e70-a20d-47ad-876e-4a47cf46325e None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:04.278741 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:04.376972 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f65e70-a20d-47ad-876e-4a47cf46325e None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:04.382688 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182444bc-50be-491d-bf51-37a4fa36b794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:04.384392 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182444bc-50be-491d-bf51-37a4fa36b794 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:04.384795 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182444bc-50be-491d-bf51-37a4fa36b794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:04.385152 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182444bc-50be-491d-bf51-37a4fa36b794 admin admin] PATH_INFO: `/v3/users/8a622e94732a429192c57963cb4e499b` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:04.386920 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182444bc-50be-491d-bf51-37a4fa36b794 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a622e94732a429192c57963cb4e499b)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:04.389261 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182444bc-50be-491d-bf51-37a4fa36b794 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:04.471101 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-182444bc-50be-491d-bf51-37a4fa36b794 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a622e94732a429192c57963cb4e499b'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:04.482673 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-182444bc-50be-491d-bf51-37a4fa36b794 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a622e94732a429192c57963cb4e499b'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:04.483436 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-182444bc-50be-491d-bf51-37a4fa36b794 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a622e94732a429192c57963cb4e499b'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:04.484063 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-182444bc-50be-491d-bf51-37a4fa36b794 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a622e94732a429192c57963cb4e499b'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:04.503259 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-182444bc-50be-491d-bf51-37a4fa36b794 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a622e94732a429192c57963cb4e499b'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:04.546730 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1091/2170] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:18:04 2022] DELETE /identity/v3/users/8a622e94732a429192c57963cb4e499b => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:18:04.554525 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0d5d3-9e44-4733-acd3-60aa8eb251dc None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:04.555512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0d5d3-9e44-4733-acd3-60aa8eb251dc None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:04.555512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0d5d3-9e44-4733-acd3-60aa8eb251dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:04.579732 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-dee0d5d3-9e44-4733-acd3-60aa8eb251dc None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:04.582002 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-dee0d5d3-9e44-4733-acd3-60aa8eb251dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:04.663047 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:04.719900 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1080/2171] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:18:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:04.730507 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182444bc-50be-491d-bf51-37a4fa36b794 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:04.732215 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-182444bc-50be-491d-bf51-37a4fa36b794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:04.822899 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182444bc-50be-491d-bf51-37a4fa36b794 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:04.828661 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e1496-4bf9-41ff-bfcf-fc6e45c51c13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:04.829703 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e1496-4bf9-41ff-bfcf-fc6e45c51c13 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:04.830105 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e1496-4bf9-41ff-bfcf-fc6e45c51c13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:04.830390 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e1496-4bf9-41ff-bfcf-fc6e45c51c13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:04.845044 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6e1496-4bf9-41ff-bfcf-fc6e45c51c13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:04.848681 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6e1496-4bf9-41ff-bfcf-fc6e45c51c13 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:04.869473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1092/2172] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:18:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:18:05.315794 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee0d5d3-9e44-4733-acd3-60aa8eb251dc None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:05.317442 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-dee0d5d3-9e44-4733-acd3-60aa8eb251dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:05.343328 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee0d5d3-9e44-4733-acd3-60aa8eb251dc None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:05.354103 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f5037-c93f-4cda-969f-a8ebd5c9d473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:05.355591 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f5037-c93f-4cda-969f-a8ebd5c9d473 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:05.356066 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f5037-c93f-4cda-969f-a8ebd5c9d473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:05.356567 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f5037-c93f-4cda-969f-a8ebd5c9d473 admin admin] PATH_INFO: `/v3/projects/fab981f8468442bc9ad854bf7930ec03` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:05.359859 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f5037-c93f-4cda-969f-a8ebd5c9d473 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fab981f8468442bc9ad854bf7930ec03)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:05.363308 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f5037-c93f-4cda-969f-a8ebd5c9d473 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:05.440387 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-805f5037-c93f-4cda-969f-a8ebd5c9d473 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fab981f8468442bc9ad854bf7930ec03', 'request_id': 'req-805f5037-c93f-4cda-969f-a8ebd5c9d473'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:05.450213 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-805f5037-c93f-4cda-969f-a8ebd5c9d473 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fab981f8468442bc9ad854bf7930ec03', 'request_id': 'req-805f5037-c93f-4cda-969f-a8ebd5c9d473'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:05.460980 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-805f5037-c93f-4cda-969f-a8ebd5c9d473 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fab981f8468442bc9ad854bf7930ec03', 'request_id': 'req-805f5037-c93f-4cda-969f-a8ebd5c9d473'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:05.476376 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-805f5037-c93f-4cda-969f-a8ebd5c9d473 admin admin] The token cache is being invalidate because project fab981f8468442bc9ad854bf7930ec03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:18:05.476881 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-805f5037-c93f-4cda-969f-a8ebd5c9d473 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:05.478457 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-805f5037-c93f-4cda-969f-a8ebd5c9d473 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:05.480563 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1081/2173] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:18:05 2022] DELETE /identity/v3/projects/fab981f8468442bc9ad854bf7930ec03 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:18:05.487726 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30030873-241a-4f49-99b9-5a7fc64dc4b2 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:05.488370 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30030873-241a-4f49-99b9-5a7fc64dc4b2 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:05.488546 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30030873-241a-4f49-99b9-5a7fc64dc4b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:05.517562 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-30030873-241a-4f49-99b9-5a7fc64dc4b2 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:05.519810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-30030873-241a-4f49-99b9-5a7fc64dc4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:05.552396 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4d217-d89e-425e-b686-8367d0ade121 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:05.552723 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4d217-d89e-425e-b686-8367d0ade121 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:05.552988 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4d217-d89e-425e-b686-8367d0ade121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:05.570289 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:05.575681 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-67b4d217-d89e-425e-b686-8367d0ade121 None None] MFA Rules not processed for user `a0408ee5961249d1bd296a6ef1a030a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:05.577055 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-67b4d217-d89e-425e-b686-8367d0ade121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:05.618717 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1093/2174] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:18:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:05.627744 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590dab99-df3b-43fa-bfcd-9acc88863a6f None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:05.628239 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590dab99-df3b-43fa-bfcd-9acc88863a6f None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:05.628783 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590dab99-df3b-43fa-bfcd-9acc88863a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:05.652253 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1082/2175] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:18:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:05.656255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-590dab99-df3b-43fa-bfcd-9acc88863a6f None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:05.657756 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-590dab99-df3b-43fa-bfcd-9acc88863a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:05.669453 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1094/2176] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:18:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:18:05.679697 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b4d217-d89e-425e-b686-8367d0ade121 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:05.693288 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b4d217-d89e-425e-b686-8367d0ade121 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:05.700333 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e09e93-83b2-4bbc-8838-b35e4bb77b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:05.701607 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e09e93-83b2-4bbc-8838-b35e4bb77b83 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:05.702516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e09e93-83b2-4bbc-8838-b35e4bb77b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:05.702884 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e09e93-83b2-4bbc-8838-b35e4bb77b83 admin admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:05.704677 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e09e93-83b2-4bbc-8838-b35e4bb77b83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:18:05.704677 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e09e93-83b2-4bbc-8838-b35e4bb77b83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:05.706206 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e09e93-83b2-4bbc-8838-b35e4bb77b83 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:05.746158 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590dab99-df3b-43fa-bfcd-9acc88863a6f None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:05.748833 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-590dab99-df3b-43fa-bfcd-9acc88863a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:05.749364 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1083/2177] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:18:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:18:05.756493 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e09e93-83b2-4bbc-8838-b35e4bb77b83 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:05.769033 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590dab99-df3b-43fa-bfcd-9acc88863a6f None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:05.778245 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e09e93-83b2-4bbc-8838-b35e4bb77b83 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:05.782443 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e34165d-2dbb-4abf-8a82-3870956b6860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:05.783562 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34165d-2dbb-4abf-8a82-3870956b6860 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:05.783956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34165d-2dbb-4abf-8a82-3870956b6860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:05.784308 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34165d-2dbb-4abf-8a82-3870956b6860 admin admin] PATH_INFO: `/v3/users/e0e1b99f79534c8a9aa10bb01b9aef9e` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:05.785528 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7a3f98-dec5-41b2-839c-9f54b409e548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:05.786219 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a3f98-dec5-41b2-839c-9f54b409e548 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:05.786412 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e34165d-2dbb-4abf-8a82-3870956b6860 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0e1b99f79534c8a9aa10bb01b9aef9e)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:05.786570 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a3f98-dec5-41b2-839c-9f54b409e548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:05.786810 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a3f98-dec5-41b2-839c-9f54b409e548 admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:05.787500 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7a3f98-dec5-41b2-839c-9f54b409e548 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:05.790792 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e34165d-2dbb-4abf-8a82-3870956b6860 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:05.792125 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7a3f98-dec5-41b2-839c-9f54b409e548 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:05.844740 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1084/2178] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:18:05 2022] POST /identity/v3/projects => generated 373 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:18:05.851243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7a3f98-dec5-41b2-839c-9f54b409e548 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:05.907724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-2e34165d-2dbb-4abf-8a82-3870956b6860 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0e1b99f79534c8a9aa10bb01b9aef9e'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:05.913095 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:05.934808 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-2e34165d-2dbb-4abf-8a82-3870956b6860 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0e1b99f79534c8a9aa10bb01b9aef9e'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:05.938328 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-2e34165d-2dbb-4abf-8a82-3870956b6860 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0e1b99f79534c8a9aa10bb01b9aef9e'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:05.939393 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-2e34165d-2dbb-4abf-8a82-3870956b6860 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0e1b99f79534c8a9aa10bb01b9aef9e'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:05.959730 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-2e34165d-2dbb-4abf-8a82-3870956b6860 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0e1b99f79534c8a9aa10bb01b9aef9e'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:05.979644 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7a3f98-dec5-41b2-839c-9f54b409e548 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:05.996836 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1cf113-f02d-4767-a48e-9ce769da8582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:05.996836 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1cf113-f02d-4767-a48e-9ce769da8582 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:05.996836 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1cf113-f02d-4767-a48e-9ce769da8582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:05.996836 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1cf113-f02d-4767-a48e-9ce769da8582 admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:05.997274 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1cf113-f02d-4767-a48e-9ce769da8582 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:05.997274 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1cf113-f02d-4767-a48e-9ce769da8582 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:06.013241 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1095/2179] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:18:05 2022] DELETE /identity/v3/users/e0e1b99f79534c8a9aa10bb01b9aef9e => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:18:06.021693 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff170b-7f6f-4401-ad36-387c0b4dcc7a None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:06.022396 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff170b-7f6f-4401-ad36-387c0b4dcc7a None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:06.023058 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff170b-7f6f-4401-ad36-387c0b4dcc7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:06.050182 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-a9ff170b-7f6f-4401-ad36-387c0b4dcc7a None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:06.050619 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1085/2180] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:18:05 2022] POST /identity/v3/users => generated 287 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:18:06.053426 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a9ff170b-7f6f-4401-ad36-387c0b4dcc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:06.053608 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1cf113-f02d-4767-a48e-9ce769da8582 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:06.057346 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-8c1cf113-f02d-4767-a48e-9ce769da8582 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:06.101262 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1cf113-f02d-4767-a48e-9ce769da8582 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:06.182833 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722c1717-4824-4451-975c-09230708ddb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:06.183581 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c1717-4824-4451-975c-09230708ddb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:06.183847 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c1717-4824-4451-975c-09230708ddb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:06.184310 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c1717-4824-4451-975c-09230708ddb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:06.185457 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722c1717-4824-4451-975c-09230708ddb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:06.187898 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722c1717-4824-4451-975c-09230708ddb8 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:06.193006 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1096/2181] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:18:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:06.210574 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff170b-7f6f-4401-ad36-387c0b4dcc7a None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:06.210882 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1086/2182] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:18:06.211077 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a9ff170b-7f6f-4401-ad36-387c0b4dcc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:06.212690 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722c1717-4824-4451-975c-09230708ddb8 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:06.226173 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722c1717-4824-4451-975c-09230708ddb8 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:06.232205 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8582f965-8df9-4747-beb7-6eb541ab2eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:06.232907 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582f965-8df9-4747-beb7-6eb541ab2eca admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:06.233239 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582f965-8df9-4747-beb7-6eb541ab2eca admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:06.233517 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582f965-8df9-4747-beb7-6eb541ab2eca admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:06.234280 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8582f965-8df9-4747-beb7-6eb541ab2eca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:06.236012 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8582f965-8df9-4747-beb7-6eb541ab2eca admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:06.255173 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1087/2183] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:18:06.256476 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8582f965-8df9-4747-beb7-6eb541ab2eca admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:06.277367 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8582f965-8df9-4747-beb7-6eb541ab2eca admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:06.282696 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41dfc2-963b-447f-b465-ae84d2a5c81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:06.284352 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41dfc2-963b-447f-b465-ae84d2a5c81f admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:06.284761 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41dfc2-963b-447f-b465-ae84d2a5c81f admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:06.285172 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41dfc2-963b-447f-b465-ae84d2a5c81f admin admin] PATH_INFO: `/v3/projects/fe9e09a9ddb84c7ea356c7df622281da/users/40ebda0ab7d045ce92da3d5512f30eaa/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:06.286525 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:06.307768 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff170b-7f6f-4401-ad36-387c0b4dcc7a None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:06.313677 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47207f4-447b-4fd7-bb9a-5b6356c95c89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:06.314685 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47207f4-447b-4fd7-bb9a-5b6356c95c89 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:06.315113 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47207f4-447b-4fd7-bb9a-5b6356c95c89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:06.315488 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47207f4-447b-4fd7-bb9a-5b6356c95c89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:06.316864 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a41dfc2-963b-447f-b465-ae84d2a5c81f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe9e09a9ddb84c7ea356c7df622281da, user_id=40ebda0ab7d045ce92da3d5512f30eaa, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:06.322195 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a41dfc2-963b-447f-b465-ae84d2a5c81f admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:06.328176 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47207f4-447b-4fd7-bb9a-5b6356c95c89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:06.330200 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47207f4-447b-4fd7-bb9a-5b6356c95c89 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:06.343361 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1097/2184] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:18:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:18:06.347880 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1088/2185] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:18:06 2022] PUT /identity/v3/projects/fe9e09a9ddb84c7ea356c7df622281da/users/40ebda0ab7d045ce92da3d5512f30eaa/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:18:06.367873 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bc49d-5e63-442c-8713-38e31602c43d None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:06.368155 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bc49d-5e63-442c-8713-38e31602c43d None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:06.368464 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bc49d-5e63-442c-8713-38e31602c43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:06.421679 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-b10bc49d-5e63-442c-8713-38e31602c43d None None] MFA Rules not processed for user `40ebda0ab7d045ce92da3d5512f30eaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:06.424877 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-b10bc49d-5e63-442c-8713-38e31602c43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:06.560995 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1098/2186] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:18:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:06.575710 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41dfc2-963b-447f-b465-ae84d2a5c81f admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:06.578824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-3a41dfc2-963b-447f-b465-ae84d2a5c81f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:06.698754 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41dfc2-963b-447f-b465-ae84d2a5c81f admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:06.704307 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4232a82c-96e9-46b4-93b5-bbbd498c3841 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:06.705180 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232a82c-96e9-46b4-93b5-bbbd498c3841 service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:06.705660 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232a82c-96e9-46b4-93b5-bbbd498c3841 service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:06.706013 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232a82c-96e9-46b4-93b5-bbbd498c3841 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:06.720720 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4232a82c-96e9-46b4-93b5-bbbd498c3841 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:06.722387 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4232a82c-96e9-46b4-93b5-bbbd498c3841 service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:06.742537 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1089/2187] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:18:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:18:06.835414 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10bc49d-5e63-442c-8713-38e31602c43d None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:06.838155 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-b10bc49d-5e63-442c-8713-38e31602c43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:06.960907 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10bc49d-5e63-442c-8713-38e31602c43d None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:06.982767 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6b6d7-325c-4dcb-bb1e-7c605bab684e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:06.984196 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6b6d7-325c-4dcb-bb1e-7c605bab684e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:06.984700 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6b6d7-325c-4dcb-bb1e-7c605bab684e admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:06.985170 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6b6d7-325c-4dcb-bb1e-7c605bab684e admin admin] PATH_INFO: `/v3/projects/a9fe138f9fd4454fbefdfc17fe392fe3` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:06.986475 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f6b6d7-325c-4dcb-bb1e-7c605bab684e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9fe138f9fd4454fbefdfc17fe392fe3)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:06.988486 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f6b6d7-325c-4dcb-bb1e-7c605bab684e admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:07.116125 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-76f6b6d7-325c-4dcb-bb1e-7c605bab684e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9fe138f9fd4454fbefdfc17fe392fe3', 'request_id': 'req-76f6b6d7-325c-4dcb-bb1e-7c605bab684e'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:07.128763 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-76f6b6d7-325c-4dcb-bb1e-7c605bab684e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9fe138f9fd4454fbefdfc17fe392fe3', 'request_id': 'req-76f6b6d7-325c-4dcb-bb1e-7c605bab684e'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:07.144139 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-76f6b6d7-325c-4dcb-bb1e-7c605bab684e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9fe138f9fd4454fbefdfc17fe392fe3', 'request_id': 'req-76f6b6d7-325c-4dcb-bb1e-7c605bab684e'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:07.171957 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-76f6b6d7-325c-4dcb-bb1e-7c605bab684e admin admin] The token cache is being invalidate because project a9fe138f9fd4454fbefdfc17fe392fe3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:18:07.174984 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-76f6b6d7-325c-4dcb-bb1e-7c605bab684e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:07.180068 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-76f6b6d7-325c-4dcb-bb1e-7c605bab684e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:07.192994 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1099/2188] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:18:06 2022] DELETE /identity/v3/projects/a9fe138f9fd4454fbefdfc17fe392fe3 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:18:08.632625 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d6304-d9cd-4638-aa91-bbaf8cf5cddc None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:08.633621 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d6304-d9cd-4638-aa91-bbaf8cf5cddc None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:08.633621 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d6304-d9cd-4638-aa91-bbaf8cf5cddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:08.662724 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-9c3d6304-d9cd-4638-aa91-bbaf8cf5cddc None None] MFA Rules not processed for user `40ebda0ab7d045ce92da3d5512f30eaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:08.664880 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-9c3d6304-d9cd-4638-aa91-bbaf8cf5cddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:08.706632 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:08.771953 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1090/2189] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:18:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:14.425163 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe8789-43f1-4523-bf80-5b00e476e87d None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:14.425737 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe8789-43f1-4523-bf80-5b00e476e87d None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:14.426085 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe8789-43f1-4523-bf80-5b00e476e87d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:14.469173 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:14.480084 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-cabe8789-43f1-4523-bf80-5b00e476e87d None None] MFA Rules not processed for user `fff9f1fc84a54b83a7a384965f40c9d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:14.482075 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-cabe8789-43f1-4523-bf80-5b00e476e87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:14.560262 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1100/2190] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:18:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:18.764177 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=15 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:24.551579 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=22 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:32.810509 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7d407-ad45-4387-bbe5-33be10646d34 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:32.811089 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7d407-ad45-4387-bbe5-33be10646d34 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:32.811089 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7d407-ad45-4387-bbe5-33be10646d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:32.852881 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-3db7d407-ad45-4387-bbe5-33be10646d34 None None] MFA Rules not processed for user `f67110d591a94537a3df6a2d835370f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:32.854125 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-3db7d407-ad45-4387-bbe5-33be10646d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:32.939507 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1091/2191] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Jun 27 21:18:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:33.008780 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe8789-43f1-4523-bf80-5b00e476e87d None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:33.013200 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-cabe8789-43f1-4523-bf80-5b00e476e87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:33.226098 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe8789-43f1-4523-bf80-5b00e476e87d None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:33.233203 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a391a5e6-1405-41d7-951a-c96483507fa6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bed6b231ee647d8a350da055d8e59d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:33.234024 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a391a5e6-1405-41d7-951a-c96483507fa6 service swift] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:33.234311 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a391a5e6-1405-41d7-951a-c96483507fa6 service swift] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:33.234585 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a391a5e6-1405-41d7-951a-c96483507fa6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:33.251349 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a391a5e6-1405-41d7-951a-c96483507fa6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:33.253489 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a391a5e6-1405-41d7-951a-c96483507fa6 service swift] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:33.263160 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181766d7-9563-4438-bb70-5100dafad686 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:33.263503 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181766d7-9563-4438-bb70-5100dafad686 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:33.263784 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181766d7-9563-4438-bb70-5100dafad686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:33.270485 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1101/2192] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Jun 27 21:18:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 21:18:33.284478 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-181766d7-9563-4438-bb70-5100dafad686 None None] MFA Rules not processed for user `f67110d591a94537a3df6a2d835370f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:33.285731 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-181766d7-9563-4438-bb70-5100dafad686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:33.294934 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1092/2193] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Jun 27 21:18:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:33.328285 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a391a5e6-1405-41d7-951a-c96483507fa6 service swift] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:33.343074 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a391a5e6-1405-41d7-951a-c96483507fa6 service swift] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:33.348555 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b1b9fc-2138-4160-8218-ad1737e2327b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bed6b231ee647d8a350da055d8e59d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:33.349285 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1b9fc-2138-4160-8218-ad1737e2327b service swift] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:33.349572 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1b9fc-2138-4160-8218-ad1737e2327b service swift] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:33.349845 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1b9fc-2138-4160-8218-ad1737e2327b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:33.360933 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b1b9fc-2138-4160-8218-ad1737e2327b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:33.363594 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b1b9fc-2138-4160-8218-ad1737e2327b service swift] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:33.372660 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1102/2194] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Jun 27 21:18:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 21:18:34.326513 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d339567-84bc-4c27-914a-02da871eda61 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:34.327492 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d339567-84bc-4c27-914a-02da871eda61 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:34.327954 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d339567-84bc-4c27-914a-02da871eda61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:34.369029 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-5d339567-84bc-4c27-914a-02da871eda61 None None] MFA Rules not processed for user `f67110d591a94537a3df6a2d835370f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:34.370429 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-5d339567-84bc-4c27-914a-02da871eda61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:34.381012 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1093/2195] 10.222.0.13 () {58 vars in 967 bytes} [Mon Jun 27 21:18:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:34.457316 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b1b9fc-2138-4160-8218-ad1737e2327b service swift] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:34.475594 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b1b9fc-2138-4160-8218-ad1737e2327b service swift] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:34.494446 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f768666-6d95-4cec-bb22-c3ff8064aaa1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bed6b231ee647d8a350da055d8e59d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:34.496012 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f768666-6d95-4cec-bb22-c3ff8064aaa1 service swift] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:34.496668 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f768666-6d95-4cec-bb22-c3ff8064aaa1 service swift] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:34.497731 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f768666-6d95-4cec-bb22-c3ff8064aaa1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:34.523827 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f768666-6d95-4cec-bb22-c3ff8064aaa1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:34.525610 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f768666-6d95-4cec-bb22-c3ff8064aaa1 service swift] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:34.536748 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1103/2196] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Jun 27 21:18:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 21:18:34.974693 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2cd1c3-999a-4180-8394-2453d3083d7b None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:34.975385 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2cd1c3-999a-4180-8394-2453d3083d7b None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:34.975771 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2cd1c3-999a-4180-8394-2453d3083d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:35.007180 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-5b2cd1c3-999a-4180-8394-2453d3083d7b None None] MFA Rules not processed for user `f67110d591a94537a3df6a2d835370f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:35.009278 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-5b2cd1c3-999a-4180-8394-2453d3083d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:35.025549 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1094/2197] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Jun 27 21:18:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:35.048199 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f768666-6d95-4cec-bb22-c3ff8064aaa1 service swift] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:35.065238 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f768666-6d95-4cec-bb22-c3ff8064aaa1 service swift] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:35.073537 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80c340-8b0b-4da9-9f1e-e651af1b62ed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bed6b231ee647d8a350da055d8e59d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:35.074531 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80c340-8b0b-4da9-9f1e-e651af1b62ed service swift] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:35.074904 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80c340-8b0b-4da9-9f1e-e651af1b62ed service swift] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:35.075225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80c340-8b0b-4da9-9f1e-e651af1b62ed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:35.084644 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc80c340-8b0b-4da9-9f1e-e651af1b62ed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:35.086796 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc80c340-8b0b-4da9-9f1e-e651af1b62ed service swift] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:35.101098 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1104/2198] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Jun 27 21:18:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 21:18:35.593982 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c06499-1e70-4669-869b-1e7958317612 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:35.594609 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c06499-1e70-4669-869b-1e7958317612 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:35.594609 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c06499-1e70-4669-869b-1e7958317612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:35.642611 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-05c06499-1e70-4669-869b-1e7958317612 None None] MFA Rules not processed for user `f67110d591a94537a3df6a2d835370f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:35.643856 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-05c06499-1e70-4669-869b-1e7958317612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:35.668027 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1095/2199] 10.222.0.13 () {58 vars in 967 bytes} [Mon Jun 27 21:18:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:35.683207 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80c340-8b0b-4da9-9f1e-e651af1b62ed service swift] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:35.712487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80c340-8b0b-4da9-9f1e-e651af1b62ed service swift] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:35.736366 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9fb494-227f-444a-8535-262b956c7402 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bed6b231ee647d8a350da055d8e59d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:35.737196 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9fb494-227f-444a-8535-262b956c7402 service swift] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:35.737502 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9fb494-227f-444a-8535-262b956c7402 service swift] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:35.740988 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9fb494-227f-444a-8535-262b956c7402 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:35.767940 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9fb494-227f-444a-8535-262b956c7402 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:35.769781 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9fb494-227f-444a-8535-262b956c7402 service swift] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:35.788011 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1105/2200] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Jun 27 21:18:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 21:18:38.232769 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c06499-1e70-4669-869b-1e7958317612 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:38.235133 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-05c06499-1e70-4669-869b-1e7958317612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:38.446066 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c06499-1e70-4669-869b-1e7958317612 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:38.453572 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cfe155-7d3d-47af-964b-d971d9bfd1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:38.454729 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfe155-7d3d-47af-964b-d971d9bfd1d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:38.455332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfe155-7d3d-47af-964b-d971d9bfd1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:38.455983 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfe155-7d3d-47af-964b-d971d9bfd1d0 admin admin] PATH_INFO: `/v3/users/40ebda0ab7d045ce92da3d5512f30eaa` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:38.457510 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cfe155-7d3d-47af-964b-d971d9bfd1d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40ebda0ab7d045ce92da3d5512f30eaa)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:38.459659 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cfe155-7d3d-47af-964b-d971d9bfd1d0 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:38.475968 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:38.537594 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-66cfe155-7d3d-47af-964b-d971d9bfd1d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40ebda0ab7d045ce92da3d5512f30eaa'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:38.547501 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-66cfe155-7d3d-47af-964b-d971d9bfd1d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40ebda0ab7d045ce92da3d5512f30eaa'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:38.548430 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-66cfe155-7d3d-47af-964b-d971d9bfd1d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40ebda0ab7d045ce92da3d5512f30eaa'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:38.549347 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-66cfe155-7d3d-47af-964b-d971d9bfd1d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40ebda0ab7d045ce92da3d5512f30eaa'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:38.567085 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-66cfe155-7d3d-47af-964b-d971d9bfd1d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40ebda0ab7d045ce92da3d5512f30eaa'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:38.595045 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1096/2201] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:18:38 2022] DELETE /identity/v3/users/40ebda0ab7d045ce92da3d5512f30eaa => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:18:38.600208 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5676167-5efb-47d5-9992-d963acc84daf None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:38.601778 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5676167-5efb-47d5-9992-d963acc84daf None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:38.602235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5676167-5efb-47d5-9992-d963acc84daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:38.635262 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-e5676167-5efb-47d5-9992-d963acc84daf None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:38.636645 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-e5676167-5efb-47d5-9992-d963acc84daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:38.714019 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:38.730232 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1106/2202] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:18:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:18:38.741577 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cfe155-7d3d-47af-964b-d971d9bfd1d0 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:38.744230 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-66cfe155-7d3d-47af-964b-d971d9bfd1d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:38.865762 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cfe155-7d3d-47af-964b-d971d9bfd1d0 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:38.869986 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d247205-3fcc-4d26-bc8a-81eec284af11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:38.870675 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d247205-3fcc-4d26-bc8a-81eec284af11 service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:38.870957 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d247205-3fcc-4d26-bc8a-81eec284af11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:38.871236 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d247205-3fcc-4d26-bc8a-81eec284af11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:38.884151 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d247205-3fcc-4d26-bc8a-81eec284af11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:38.885917 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d247205-3fcc-4d26-bc8a-81eec284af11 service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:38.905237 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1097/2203] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:18:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:18:39.133744 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5676167-5efb-47d5-9992-d963acc84daf None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:39.136015 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-e5676167-5efb-47d5-9992-d963acc84daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:39.166141 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5676167-5efb-47d5-9992-d963acc84daf None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:39.166770 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002305da-2081-4e24-ad49-d749ad5e7fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:39.167878 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002305da-2081-4e24-ad49-d749ad5e7fb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:39.168600 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002305da-2081-4e24-ad49-d749ad5e7fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:39.168600 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002305da-2081-4e24-ad49-d749ad5e7fb4 admin admin] PATH_INFO: `/v3/projects/fe9e09a9ddb84c7ea356c7df622281da` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:39.170172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002305da-2081-4e24-ad49-d749ad5e7fb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe9e09a9ddb84c7ea356c7df622281da)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:39.174014 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002305da-2081-4e24-ad49-d749ad5e7fb4 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:39.249430 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-002305da-2081-4e24-ad49-d749ad5e7fb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe9e09a9ddb84c7ea356c7df622281da', 'request_id': 'req-002305da-2081-4e24-ad49-d749ad5e7fb4'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:39.257201 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-002305da-2081-4e24-ad49-d749ad5e7fb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe9e09a9ddb84c7ea356c7df622281da', 'request_id': 'req-002305da-2081-4e24-ad49-d749ad5e7fb4'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:39.281514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-002305da-2081-4e24-ad49-d749ad5e7fb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe9e09a9ddb84c7ea356c7df622281da', 'request_id': 'req-002305da-2081-4e24-ad49-d749ad5e7fb4'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:39.307374 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-002305da-2081-4e24-ad49-d749ad5e7fb4 admin admin] The token cache is being invalidate because project fe9e09a9ddb84c7ea356c7df622281da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:18:39.307886 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-002305da-2081-4e24-ad49-d749ad5e7fb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:39.309072 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-002305da-2081-4e24-ad49-d749ad5e7fb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:18:39.311177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1107/2204] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:18:39 2022] DELETE /identity/v3/projects/fe9e09a9ddb84c7ea356c7df622281da => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:18:41.729218 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d247205-3fcc-4d26-bc8a-81eec284af11 service neutron] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:41.730685 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-5d247205-3fcc-4d26-bc8a-81eec284af11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:41.745204 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d247205-3fcc-4d26-bc8a-81eec284af11 service neutron] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:41.813405 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63903082-7a21-46ad-b75f-9fdd1448d6f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:41.814337 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63903082-7a21-46ad-b75f-9fdd1448d6f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:41.814858 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63903082-7a21-46ad-b75f-9fdd1448d6f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:41.815162 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63903082-7a21-46ad-b75f-9fdd1448d6f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:41.818550 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-63903082-7a21-46ad-b75f-9fdd1448d6f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:41.879811 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:41.977531 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63903082-7a21-46ad-b75f-9fdd1448d6f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:41.980331 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63903082-7a21-46ad-b75f-9fdd1448d6f9 service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:42.017022 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1098/2205] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:18:41 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:18:48.682481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5278c0b-247e-4882-ae5e-078ddfc83303 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:48.683406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5278c0b-247e-4882-ae5e-078ddfc83303 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:48.683406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5278c0b-247e-4882-ae5e-078ddfc83303 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:48.684533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1108/2206] 10.222.0.13 () {54 vars in 921 bytes} [Mon Jun 27 21:18:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:18:48.690856 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e7194-db98-4555-9adf-6c87287aaa35 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:48.691467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e7194-db98-4555-9adf-6c87287aaa35 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:48.691467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e7194-db98-4555-9adf-6c87287aaa35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:48.764226 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-4c0e7194-db98-4555-9adf-6c87287aaa35 None None] MFA Rules not processed for user `1366958034d540b4ba2feb421a6afabc`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:48.765499 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-4c0e7194-db98-4555-9adf-6c87287aaa35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:48.865983 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1099/2207] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Jun 27 21:18:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:48.880502 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5278c0b-247e-4882-ae5e-078ddfc83303 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:48.882490 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-e5278c0b-247e-4882-ae5e-078ddfc83303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:48.956286 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5278c0b-247e-4882-ae5e-078ddfc83303 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:48.962601 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0380995d-fe06-4e2f-9a5b-aeaf5b82b9f3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26cf962d151b4da7a82c33ef23b342e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:48.963433 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380995d-fe06-4e2f-9a5b-aeaf5b82b9f3 service glance] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:48.963717 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380995d-fe06-4e2f-9a5b-aeaf5b82b9f3 service glance] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:48.963995 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380995d-fe06-4e2f-9a5b-aeaf5b82b9f3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:48.977086 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0380995d-fe06-4e2f-9a5b-aeaf5b82b9f3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:48.978592 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0380995d-fe06-4e2f-9a5b-aeaf5b82b9f3 service glance] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:48.991416 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1109/2208] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:18:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:18:49.066416 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f1cee-9288-4b0d-a3ec-599cab05a4ed None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:49.066723 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f1cee-9288-4b0d-a3ec-599cab05a4ed None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:49.067123 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f1cee-9288-4b0d-a3ec-599cab05a4ed None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:49.068607 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1100/2209] 10.222.0.13 () {54 vars in 921 bytes} [Mon Jun 27 21:18:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:18:49.075993 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9b2b2-359b-41e5-8f40-53a94dc37097 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:49.076347 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9b2b2-359b-41e5-8f40-53a94dc37097 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:49.076670 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9b2b2-359b-41e5-8f40-53a94dc37097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:49.099722 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-13a9b2b2-359b-41e5-8f40-53a94dc37097 None None] MFA Rules not processed for user `2a4e105858324391b5142d57e34aff9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:49.101648 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-13a9b2b2-359b-41e5-8f40-53a94dc37097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:49.112124 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1110/2210] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Jun 27 21:18:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:18:49.124966 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f1cee-9288-4b0d-a3ec-599cab05a4ed None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:49.127127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-076f1cee-9288-4b0d-a3ec-599cab05a4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:49.141055 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f1cee-9288-4b0d-a3ec-599cab05a4ed None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:49.144712 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d570bcd4-b25d-488c-a333-a4e9a4958bf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:49.145395 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d570bcd4-b25d-488c-a333-a4e9a4958bf7 service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:49.145781 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d570bcd4-b25d-488c-a333-a4e9a4958bf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:49.146060 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d570bcd4-b25d-488c-a333-a4e9a4958bf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:49.155301 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d570bcd4-b25d-488c-a333-a4e9a4958bf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:49.156998 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d570bcd4-b25d-488c-a333-a4e9a4958bf7 service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:49.166960 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1101/2211] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:18:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:18:49.259996 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2284b1d0-4ea9-48ac-875b-9ac92196fa1b None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:49.260307 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2284b1d0-4ea9-48ac-875b-9ac92196fa1b None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:49.260580 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2284b1d0-4ea9-48ac-875b-9ac92196fa1b None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:49.262005 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1111/2212] 10.222.0.13 () {54 vars in 921 bytes} [Mon Jun 27 21:18:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:18:49.268030 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7676bac-6fe9-48e2-bb40-841491a4fb6d None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:49.269275 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7676bac-6fe9-48e2-bb40-841491a4fb6d None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:49.269566 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7676bac-6fe9-48e2-bb40-841491a4fb6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:49.293350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-e7676bac-6fe9-48e2-bb40-841491a4fb6d None None] MFA Rules not processed for user `1366958034d540b4ba2feb421a6afabc`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:49.294728 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-e7676bac-6fe9-48e2-bb40-841491a4fb6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:49.306086 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1102/2213] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Jun 27 21:18:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:49.317968 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2284b1d0-4ea9-48ac-875b-9ac92196fa1b None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:49.320518 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-2284b1d0-4ea9-48ac-875b-9ac92196fa1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:49.335092 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2284b1d0-4ea9-48ac-875b-9ac92196fa1b None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:49.339365 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6743e5d-03fb-4a56-b18b-a52ef81d022a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:49.340020 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6743e5d-03fb-4a56-b18b-a52ef81d022a service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:49.340301 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6743e5d-03fb-4a56-b18b-a52ef81d022a service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:49.340579 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6743e5d-03fb-4a56-b18b-a52ef81d022a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:49.353014 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6743e5d-03fb-4a56-b18b-a52ef81d022a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:49.354614 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6743e5d-03fb-4a56-b18b-a52ef81d022a service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:49.367498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1112/2214] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:18:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:18:52.695208 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01bd3cf-2ebc-4220-8cf1-0ebcda00065e None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:52.695780 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01bd3cf-2ebc-4220-8cf1-0ebcda00065e None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:52.695927 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01bd3cf-2ebc-4220-8cf1-0ebcda00065e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:52.717900 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-b01bd3cf-2ebc-4220-8cf1-0ebcda00065e None None] MFA Rules not processed for user `f67110d591a94537a3df6a2d835370f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:18:52.719138 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-b01bd3cf-2ebc-4220-8cf1-0ebcda00065e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:52.771184 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:52.787136 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1103/2215] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Jun 27 21:18:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:18:52.800979 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6743e5d-03fb-4a56-b18b-a52ef81d022a service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:18:52.802792 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a6743e5d-03fb-4a56-b18b-a52ef81d022a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:18:52.891533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6743e5d-03fb-4a56-b18b-a52ef81d022a service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:18:52.896773 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64db91a-17b9-425b-8399-54f87d7b423d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bed6b231ee647d8a350da055d8e59d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:18:52.897482 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64db91a-17b9-425b-8399-54f87d7b423d service swift] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:18:52.897853 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64db91a-17b9-425b-8399-54f87d7b423d service swift] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:18:52.898778 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64db91a-17b9-425b-8399-54f87d7b423d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:18:52.904718 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:18:52.909616 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64db91a-17b9-425b-8399-54f87d7b423d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:18:52.911776 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64db91a-17b9-425b-8399-54f87d7b423d service swift] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:18:52.923019 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1113/2216] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Jun 27 21:18:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 21:19:02.778163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=3 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:19:02.924530 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:19:18.727593 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec776d-17bd-4e45-88b5-0bb433a50d51 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:19:18.728255 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec776d-17bd-4e45-88b5-0bb433a50d51 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:19:18.728399 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec776d-17bd-4e45-88b5-0bb433a50d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:19:18.792520 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-a3ec776d-17bd-4e45-88b5-0bb433a50d51 None None] MFA Rules not processed for user `f67110d591a94537a3df6a2d835370f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:19:18.793834 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-a3ec776d-17bd-4e45-88b5-0bb433a50d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:19:18.815136 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1104/2217] 10.222.0.13 () {58 vars in 967 bytes} [Mon Jun 27 21:19:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:19:18.828812 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64db91a-17b9-425b-8399-54f87d7b423d service swift] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:19:18.857428 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64db91a-17b9-425b-8399-54f87d7b423d service swift] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:19:18.863298 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b67ca5b-c51a-42e9-9b95-39b1fc625684 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bed6b231ee647d8a350da055d8e59d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:19:18.863996 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67ca5b-c51a-42e9-9b95-39b1fc625684 service swift] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:19:18.864301 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67ca5b-c51a-42e9-9b95-39b1fc625684 service swift] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:19:18.864570 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67ca5b-c51a-42e9-9b95-39b1fc625684 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:19:18.874510 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b67ca5b-c51a-42e9-9b95-39b1fc625684 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:19:18.876145 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b67ca5b-c51a-42e9-9b95-39b1fc625684 service swift] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:19:18.886483 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1114/2218] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Jun 27 21:19:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 21:19:24.438345 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec776d-17bd-4e45-88b5-0bb433a50d51 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:19:24.439890 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-a3ec776d-17bd-4e45-88b5-0bb433a50d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:19:24.571055 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec776d-17bd-4e45-88b5-0bb433a50d51 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:19:24.582684 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d81f858-2939-4389-ac5e-354a759c34b3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0437c45d72cd4a38b9d48ee6f50946fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:19:24.583937 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81f858-2939-4389-ac5e-354a759c34b3 service placement] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:19:24.585241 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81f858-2939-4389-ac5e-354a759c34b3 service placement] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:19:24.585241 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81f858-2939-4389-ac5e-354a759c34b3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:19:24.587917 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-0d81f858-2939-4389-ac5e-354a759c34b3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:19:24.603221 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d81f858-2939-4389-ac5e-354a759c34b3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:19:24.605200 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d81f858-2939-4389-ac5e-354a759c34b3 service placement] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:19:24.618149 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1105/2219] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:19:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:19:28.882451 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=10 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:19:34.612816 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=59 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:19:34.971386 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36013ee2-b6b7-41fe-8b52-b784bcb7b8f5 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:19:34.971917 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36013ee2-b6b7-41fe-8b52-b784bcb7b8f5 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:19:34.972014 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36013ee2-b6b7-41fe-8b52-b784bcb7b8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:19:34.990487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-36013ee2-b6b7-41fe-8b52-b784bcb7b8f5 None None] MFA Rules not processed for user `fff9f1fc84a54b83a7a384965f40c9d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:19:34.991701 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-36013ee2-b6b7-41fe-8b52-b784bcb7b8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:19:35.050848 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1115/2220] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:19:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:19:43.085630 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2dde50-887e-48b7-8bf9-bf0e117771f7 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:19:43.086349 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2dde50-887e-48b7-8bf9-bf0e117771f7 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:19:43.086349 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2dde50-887e-48b7-8bf9-bf0e117771f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:19:43.105290 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-4e2dde50-887e-48b7-8bf9-bf0e117771f7 None None] MFA Rules not processed for user `a0408ee5961249d1bd296a6ef1a030a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:19:43.106522 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-4e2dde50-887e-48b7-8bf9-bf0e117771f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:19:43.166566 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1106/2221] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:19:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:19:45.044077 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=25 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:19:53.159177 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=25 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:20:06.433184 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f30274-8a69-4f90-a071-ef12a5a2cf1a None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:20:06.433849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f30274-8a69-4f90-a071-ef12a5a2cf1a None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:20:06.433849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f30274-8a69-4f90-a071-ef12a5a2cf1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:20:06.453664 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-a0f30274-8a69-4f90-a071-ef12a5a2cf1a None None] MFA Rules not processed for user `ba0127dd69da407ab096773fc9e22530`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:20:06.454895 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a0f30274-8a69-4f90-a071-ef12a5a2cf1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:20:06.515273 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1116/2222] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:20:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:20:16.507440 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=25 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:20:23.124837 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e64a2e-54b1-4319-a38c-6c29ed0f42ca None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:20:23.125483 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e64a2e-54b1-4319-a38c-6c29ed0f42ca None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:20:23.125741 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e64a2e-54b1-4319-a38c-6c29ed0f42ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:20:23.146242 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-08e64a2e-54b1-4319-a38c-6c29ed0f42ca None None] MFA Rules not processed for user `7969c6620a0d406890fb4c15b58ab349`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:20:23.147473 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-08e64a2e-54b1-4319-a38c-6c29ed0f42ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:20:23.208963 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1107/2223] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:20:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:20:33.200450 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=25 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:20:50.225525 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f30274-8a69-4f90-a071-ef12a5a2cf1a None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:20:50.236260 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f30274-8a69-4f90-a071-ef12a5a2cf1a None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:20:50.240298 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6971e7b3-1fb4-46c3-ac3f-e01a701987a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:20:50.241067 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6971e7b3-1fb4-46c3-ac3f-e01a701987a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:20:50.246699 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6971e7b3-1fb4-46c3-ac3f-e01a701987a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:20:50.246699 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6971e7b3-1fb4-46c3-ac3f-e01a701987a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:20:50.246699 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-6971e7b3-1fb4-46c3-ac3f-e01a701987a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:20:50.263610 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6971e7b3-1fb4-46c3-ac3f-e01a701987a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:20:50.265105 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6971e7b3-1fb4-46c3-ac3f-e01a701987a5 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:20:50.277716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1117/2224] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:20:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:20:51.879634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e64a2e-54b1-4319-a38c-6c29ed0f42ca None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:20:51.891165 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e64a2e-54b1-4319-a38c-6c29ed0f42ca None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:20:51.895202 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f289e492-7552-4b48-b778-a5a6269855d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:20:51.896089 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289e492-7552-4b48-b778-a5a6269855d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:20:51.896435 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289e492-7552-4b48-b778-a5a6269855d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:20:51.896917 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289e492-7552-4b48-b778-a5a6269855d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:20:51.899245 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-f289e492-7552-4b48-b778-a5a6269855d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:20:51.996735 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f289e492-7552-4b48-b778-a5a6269855d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:20:51.998252 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f289e492-7552-4b48-b778-a5a6269855d5 service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:20:52.015030 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1108/2225] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:20:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:20:55.953832 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6971e7b3-1fb4-46c3-ac3f-e01a701987a5 service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:20:55.963647 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6971e7b3-1fb4-46c3-ac3f-e01a701987a5 service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:20:55.969360 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3fda3b-c864-4509-898c-e8af31822092 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:20:55.971078 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3fda3b-c864-4509-898c-e8af31822092 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:20:55.971476 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3fda3b-c864-4509-898c-e8af31822092 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:20:55.971999 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3fda3b-c864-4509-898c-e8af31822092 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:20:55.975288 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-4b3fda3b-c864-4509-898c-e8af31822092 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:20:55.996865 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3fda3b-c864-4509-898c-e8af31822092 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:20:55.998678 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3fda3b-c864-4509-898c-e8af31822092 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:20:56.016873 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1118/2226] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:20:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:20:56.683135 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e6743-9dd3-4b3a-9cf5-c6e5e5215c30 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:20:56.684037 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e6743-9dd3-4b3a-9cf5-c6e5e5215c30 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:20:56.684037 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e6743-9dd3-4b3a-9cf5-c6e5e5215c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:20:56.707354 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-203e6743-9dd3-4b3a-9cf5-c6e5e5215c30 None None] MFA Rules not processed for user `fff9f1fc84a54b83a7a384965f40c9d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:20:56.708707 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-203e6743-9dd3-4b3a-9cf5-c6e5e5215c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:20:56.722122 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1109/2227] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:20:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:20:57.190914 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231e7c6-2b0f-42ce-afa2-91a146cded52 None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:20:57.192742 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231e7c6-2b0f-42ce-afa2-91a146cded52 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:20:57.193357 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231e7c6-2b0f-42ce-afa2-91a146cded52 None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:20:57.196191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1119/2228] 10.222.0.13 () {54 vars in 921 bytes} [Mon Jun 27 21:20:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 27 21:20:57.212220 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838b065-3815-408e-acfa-c9c6679dc605 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:20:57.213218 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838b065-3815-408e-acfa-c9c6679dc605 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:20:57.213694 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838b065-3815-408e-acfa-c9c6679dc605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:20:57.263835 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-f838b065-3815-408e-acfa-c9c6679dc605 None None] MFA Rules not processed for user `0437c45d72cd4a38b9d48ee6f50946fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:20:57.265726 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-f838b065-3815-408e-acfa-c9c6679dc605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:20:57.278924 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1110/2229] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Jun 27 21:20:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:20:57.292627 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231e7c6-2b0f-42ce-afa2-91a146cded52 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:20:57.311416 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231e7c6-2b0f-42ce-afa2-91a146cded52 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:20:57.317505 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf86e28-c32c-41a7-8477-22336541f2ca service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0437c45d72cd4a38b9d48ee6f50946fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:20:57.318177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf86e28-c32c-41a7-8477-22336541f2ca service placement] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:20:57.318439 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf86e28-c32c-41a7-8477-22336541f2ca service placement] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:20:57.318996 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf86e28-c32c-41a7-8477-22336541f2ca service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:20:57.333688 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf86e28-c32c-41a7-8477-22336541f2ca service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:20:57.337127 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf86e28-c32c-41a7-8477-22336541f2ca service placement] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:20:57.358348 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f838b065-3815-408e-acfa-c9c6679dc605 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:20:57.359662 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-f838b065-3815-408e-acfa-c9c6679dc605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:20:57.390188 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1120/2230] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:20:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:20:57.410996 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f838b065-3815-408e-acfa-c9c6679dc605 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:20:57.429376 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3194e58e-6813-419c-9cc0-a5c4afb3d4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:20:57.431542 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194e58e-6813-419c-9cc0-a5c4afb3d4e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:20:57.431999 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194e58e-6813-419c-9cc0-a5c4afb3d4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:20:57.432547 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194e58e-6813-419c-9cc0-a5c4afb3d4e7 admin admin] PATH_INFO: `/v3/users/a0408ee5961249d1bd296a6ef1a030a9` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:20:57.436786 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3194e58e-6813-419c-9cc0-a5c4afb3d4e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0408ee5961249d1bd296a6ef1a030a9)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:20:57.444289 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3194e58e-6813-419c-9cc0-a5c4afb3d4e7 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:20:57.545270 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-3194e58e-6813-419c-9cc0-a5c4afb3d4e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0408ee5961249d1bd296a6ef1a030a9'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:20:57.558556 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-3194e58e-6813-419c-9cc0-a5c4afb3d4e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0408ee5961249d1bd296a6ef1a030a9'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:20:57.559606 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-3194e58e-6813-419c-9cc0-a5c4afb3d4e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0408ee5961249d1bd296a6ef1a030a9'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:20:57.560522 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-3194e58e-6813-419c-9cc0-a5c4afb3d4e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0408ee5961249d1bd296a6ef1a030a9'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:20:57.572134 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-3194e58e-6813-419c-9cc0-a5c4afb3d4e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0408ee5961249d1bd296a6ef1a030a9'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:20:57.597056 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1111/2231] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:20:57 2022] DELETE /identity/v3/users/a0408ee5961249d1bd296a6ef1a030a9 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:20:57.932196 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf86e28-c32c-41a7-8477-22336541f2ca service placement] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:20:57.964705 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf86e28-c32c-41a7-8477-22336541f2ca service placement] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:20:58.047365 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb0dbb-0753-424d-a6c9-d42aa01f9042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:20:58.048418 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb0dbb-0753-424d-a6c9-d42aa01f9042 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:20:58.048794 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb0dbb-0753-424d-a6c9-d42aa01f9042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:20:58.049591 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb0dbb-0753-424d-a6c9-d42aa01f9042 admin admin] PATH_INFO: `/v3/projects/a2a3853bab51428eb89ff228e5f29ef4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:20:58.051296 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cb0dbb-0753-424d-a6c9-d42aa01f9042 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2a3853bab51428eb89ff228e5f29ef4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:20:58.054476 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cb0dbb-0753-424d-a6c9-d42aa01f9042 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:20:58.123373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-80cb0dbb-0753-424d-a6c9-d42aa01f9042 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2a3853bab51428eb89ff228e5f29ef4', 'request_id': 'req-80cb0dbb-0753-424d-a6c9-d42aa01f9042'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:20:58.130894 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-80cb0dbb-0753-424d-a6c9-d42aa01f9042 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2a3853bab51428eb89ff228e5f29ef4', 'request_id': 'req-80cb0dbb-0753-424d-a6c9-d42aa01f9042'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:20:58.139608 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-80cb0dbb-0753-424d-a6c9-d42aa01f9042 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2a3853bab51428eb89ff228e5f29ef4', 'request_id': 'req-80cb0dbb-0753-424d-a6c9-d42aa01f9042'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:20:58.151127 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-80cb0dbb-0753-424d-a6c9-d42aa01f9042 admin admin] The token cache is being invalidate because project a2a3853bab51428eb89ff228e5f29ef4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:20:58.151441 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-80cb0dbb-0753-424d-a6c9-d42aa01f9042 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:20:58.152067 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-80cb0dbb-0753-424d-a6c9-d42aa01f9042 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:20:58.158681 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1121/2232] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:20:57 2022] DELETE /identity/v3/projects/a2a3853bab51428eb89ff228e5f29ef4 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:20:58.160349 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd40d71-cb8f-4f7e-8eab-638be32fa3ef None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:20:58.161153 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd40d71-cb8f-4f7e-8eab-638be32fa3ef None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:20:58.161153 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd40d71-cb8f-4f7e-8eab-638be32fa3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:20:58.208642 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-edd40d71-cb8f-4f7e-8eab-638be32fa3ef None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:20:58.210156 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-edd40d71-cb8f-4f7e-8eab-638be32fa3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:20:58.230172 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:20:58.282500 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1112/2233] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:20:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:20:58.291972 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc2139-b09e-42e7-9856-51c32451710f None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:20:58.292881 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc2139-b09e-42e7-9856-51c32451710f None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:20:58.293092 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc2139-b09e-42e7-9856-51c32451710f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:20:58.315673 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-f7dc2139-b09e-42e7-9856-51c32451710f None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:20:58.317054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-f7dc2139-b09e-42e7-9856-51c32451710f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:20:58.325946 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1122/2234] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:20:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:20:58.334906 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd40d71-cb8f-4f7e-8eab-638be32fa3ef None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:20:58.349803 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd40d71-cb8f-4f7e-8eab-638be32fa3ef None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:20:58.355553 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d5b49e-9dbb-4f91-a5c3-01c6bda55acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:20:58.356251 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5b49e-9dbb-4f91-a5c3-01c6bda55acc admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:20:58.356532 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5b49e-9dbb-4f91-a5c3-01c6bda55acc admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:20:58.356813 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5b49e-9dbb-4f91-a5c3-01c6bda55acc admin admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:20:58.357588 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d5b49e-9dbb-4f91-a5c3-01c6bda55acc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:20:58.358039 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d5b49e-9dbb-4f91-a5c3-01c6bda55acc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:20:58.359700 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d5b49e-9dbb-4f91-a5c3-01c6bda55acc admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:20:58.389050 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1113/2235] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:20:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:20:58.392448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc2139-b09e-42e7-9856-51c32451710f None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:20:58.410062 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc2139-b09e-42e7-9856-51c32451710f None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:20:58.417823 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4c1ff-61fe-4b63-886f-edb92aa40cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:20:58.418727 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4c1ff-61fe-4b63-886f-edb92aa40cc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:20:58.419294 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4c1ff-61fe-4b63-886f-edb92aa40cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:20:58.419655 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4c1ff-61fe-4b63-886f-edb92aa40cc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:20:58.420612 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f4c1ff-61fe-4b63-886f-edb92aa40cc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:20:58.422770 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f4c1ff-61fe-4b63-886f-edb92aa40cc8 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:20:58.457193 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1123/2236] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:20:58 2022] POST /identity/v3/projects => generated 403 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:20:58.462259 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d5b49e-9dbb-4f91-a5c3-01c6bda55acc admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:20:58.473063 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d5b49e-9dbb-4f91-a5c3-01c6bda55acc admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:20:58.554590 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b14fcf-b0aa-48e4-897a-a4b7b6d2b4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:20:58.555544 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b14fcf-b0aa-48e4-897a-a4b7b6d2b4af admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:20:58.556340 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b14fcf-b0aa-48e4-897a-a4b7b6d2b4af admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:20:58.556759 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b14fcf-b0aa-48e4-897a-a4b7b6d2b4af admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:20:58.557594 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b14fcf-b0aa-48e4-897a-a4b7b6d2b4af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:20:58.559856 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b14fcf-b0aa-48e4-897a-a4b7b6d2b4af admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:20:58.585100 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1114/2237] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:20:58 2022] POST /identity/v3/users => generated 302 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:20:58.591068 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4c1ff-61fe-4b63-886f-edb92aa40cc8 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:20:58.607904 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4c1ff-61fe-4b63-886f-edb92aa40cc8 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:20:58.613373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa1c75-583f-40d9-b437-8676f03feae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:20:58.614062 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa1c75-583f-40d9-b437-8676f03feae6 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:20:58.614330 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa1c75-583f-40d9-b437-8676f03feae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:20:58.614665 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa1c75-583f-40d9-b437-8676f03feae6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:20:58.615436 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fa1c75-583f-40d9-b437-8676f03feae6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:20:58.617310 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fa1c75-583f-40d9-b437-8676f03feae6 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:20:58.629406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1124/2238] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:20:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:20:58.634339 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b14fcf-b0aa-48e4-897a-a4b7b6d2b4af admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:20:58.646682 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b14fcf-b0aa-48e4-897a-a4b7b6d2b4af admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:20:58.652293 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1472d54-2566-4cad-90a1-d0761665b906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:20:58.653081 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1472d54-2566-4cad-90a1-d0761665b906 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:20:58.653453 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1472d54-2566-4cad-90a1-d0761665b906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:20:58.653770 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1472d54-2566-4cad-90a1-d0761665b906 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:20:58.654709 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1472d54-2566-4cad-90a1-d0761665b906 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:20:58.656455 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1472d54-2566-4cad-90a1-d0761665b906 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:20:58.669371 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1115/2239] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:20:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:20:58.675041 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa1c75-583f-40d9-b437-8676f03feae6 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:20:58.686049 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa1c75-583f-40d9-b437-8676f03feae6 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:20:58.692413 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b0f425-d197-4458-b92b-1baedf6b1177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:20:58.693465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0f425-d197-4458-b92b-1baedf6b1177 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:20:58.693808 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0f425-d197-4458-b92b-1baedf6b1177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:20:58.694136 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0f425-d197-4458-b92b-1baedf6b1177 admin admin] PATH_INFO: `/v3/projects/6370ee04d3a14bb2a59bde4e79c70db6/users/f50635e25f7743c38cd2cf2fefff39c1/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:20:58.703958 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:20:58.714199 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b0f425-d197-4458-b92b-1baedf6b1177 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6370ee04d3a14bb2a59bde4e79c70db6, user_id=f50635e25f7743c38cd2cf2fefff39c1, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:20:58.717684 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b0f425-d197-4458-b92b-1baedf6b1177 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:20:58.736260 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1125/2240] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:20:58 2022] PUT /identity/v3/projects/6370ee04d3a14bb2a59bde4e79c70db6/users/f50635e25f7743c38cd2cf2fefff39c1/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:20:58.741758 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aef341-7741-4104-9924-3e06e78b0253 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:20:58.742172 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aef341-7741-4104-9924-3e06e78b0253 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:20:58.742622 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aef341-7741-4104-9924-3e06e78b0253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:20:58.764197 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-99aef341-7741-4104-9924-3e06e78b0253 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:20:58.765657 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-99aef341-7741-4104-9924-3e06e78b0253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:20:58.832312 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1116/2241] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:20:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:20:58.844251 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b0f425-d197-4458-b92b-1baedf6b1177 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:20:58.846395 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a4b0f425-d197-4458-b92b-1baedf6b1177 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:20:58.909471 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b0f425-d197-4458-b92b-1baedf6b1177 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:20:58.913505 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17ab2c-12af-49f6-a462-c89963f7644b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:20:58.914179 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17ab2c-12af-49f6-a462-c89963f7644b service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:20:58.914467 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17ab2c-12af-49f6-a462-c89963f7644b service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:20:58.914732 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17ab2c-12af-49f6-a462-c89963f7644b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:20:58.926247 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a17ab2c-12af-49f6-a462-c89963f7644b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:20:58.928014 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a17ab2c-12af-49f6-a462-c89963f7644b service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:20:58.940683 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1126/2242] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:20:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:21:00.001941 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c42118-6326-416e-8ab5-b9ee0d33b25b None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:00.002503 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c42118-6326-416e-8ab5-b9ee0d33b25b None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:00.002759 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c42118-6326-416e-8ab5-b9ee0d33b25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:00.067455 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-b2c42118-6326-416e-8ab5-b9ee0d33b25b None None] MFA Rules not processed for user `f50635e25f7743c38cd2cf2fefff39c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:21:00.069675 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-b2c42118-6326-416e-8ab5-b9ee0d33b25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:00.108904 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:00.247079 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1117/2243] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:20:59 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:21:00.255679 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17ab2c-12af-49f6-a462-c89963f7644b service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:00.283662 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17ab2c-12af-49f6-a462-c89963f7644b service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:00.289239 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebada63-dd42-4b70-801c-361a6f3f2a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:00.289954 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebada63-dd42-4b70-801c-361a6f3f2a43 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:00.291301 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebada63-dd42-4b70-801c-361a6f3f2a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:00.291301 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebada63-dd42-4b70-801c-361a6f3f2a43 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:00.291716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebada63-dd42-4b70-801c-361a6f3f2a43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:00.293737 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebada63-dd42-4b70-801c-361a6f3f2a43 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:00.330514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1127/2244] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:21:00 2022] POST /identity/v3/projects => generated 405 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:21:00.336813 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c42118-6326-416e-8ab5-b9ee0d33b25b None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:00.356530 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c42118-6326-416e-8ab5-b9ee0d33b25b None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:00.484837 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4488d43-daa8-41a7-bd16-cb21f2abe49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:00.485946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4488d43-daa8-41a7-bd16-cb21f2abe49d admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:00.486298 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4488d43-daa8-41a7-bd16-cb21f2abe49d admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:00.486634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4488d43-daa8-41a7-bd16-cb21f2abe49d admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:00.487424 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4488d43-daa8-41a7-bd16-cb21f2abe49d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:00.490127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4488d43-daa8-41a7-bd16-cb21f2abe49d admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:00.532506 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1118/2245] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:21:00 2022] POST /identity/v3/users => generated 309 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:21:00.538787 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebada63-dd42-4b70-801c-361a6f3f2a43 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:00.552478 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebada63-dd42-4b70-801c-361a6f3f2a43 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:00.559482 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b895f00-94ee-4eb8-8c5a-5e5ade4ab448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:00.561059 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b895f00-94ee-4eb8-8c5a-5e5ade4ab448 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:00.561633 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b895f00-94ee-4eb8-8c5a-5e5ade4ab448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:00.562222 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b895f00-94ee-4eb8-8c5a-5e5ade4ab448 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:00.563849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b895f00-94ee-4eb8-8c5a-5e5ade4ab448 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:00.567176 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b895f00-94ee-4eb8-8c5a-5e5ade4ab448 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:00.589950 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1128/2246] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:21:00 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:21:00.595370 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4488d43-daa8-41a7-bd16-cb21f2abe49d admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:00.612070 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4488d43-daa8-41a7-bd16-cb21f2abe49d admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:00.617851 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e8388-fdc2-4165-b6c2-a3da9e1ce766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:00.619096 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e8388-fdc2-4165-b6c2-a3da9e1ce766 admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:00.619721 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e8388-fdc2-4165-b6c2-a3da9e1ce766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:00.620161 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e8388-fdc2-4165-b6c2-a3da9e1ce766 admin admin] PATH_INFO: `/v3/projects/5365d63a880842feb79fc933412444c5/users/6625e1b1d5164bea8cb0010becb1e7d2/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:00.639017 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e8388-fdc2-4165-b6c2-a3da9e1ce766 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5365d63a880842feb79fc933412444c5, user_id=6625e1b1d5164bea8cb0010becb1e7d2, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:00.641986 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e8388-fdc2-4165-b6c2-a3da9e1ce766 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:00.659953 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1119/2247] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:21:00 2022] PUT /identity/v3/projects/5365d63a880842feb79fc933412444c5/users/6625e1b1d5164bea8cb0010becb1e7d2/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:21:00.664257 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b895f00-94ee-4eb8-8c5a-5e5ade4ab448 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:00.677977 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b895f00-94ee-4eb8-8c5a-5e5ade4ab448 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:00.771638 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc60a85a-8da1-4ca4-a209-9416a5d62038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:00.772838 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60a85a-8da1-4ca4-a209-9416a5d62038 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:00.773375 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60a85a-8da1-4ca4-a209-9416a5d62038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:00.773840 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60a85a-8da1-4ca4-a209-9416a5d62038 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:00.775311 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc60a85a-8da1-4ca4-a209-9416a5d62038 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:00.778338 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc60a85a-8da1-4ca4-a209-9416a5d62038 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:00.800504 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1129/2248] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:21:00 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:21:00.802325 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e8388-fdc2-4165-b6c2-a3da9e1ce766 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:00.815230 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e8388-fdc2-4165-b6c2-a3da9e1ce766 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:00.820080 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662443d-454b-4652-935c-f111f3dd2839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:00.821007 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662443d-454b-4652-935c-f111f3dd2839 admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:00.821496 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662443d-454b-4652-935c-f111f3dd2839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:00.822108 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662443d-454b-4652-935c-f111f3dd2839 admin admin] PATH_INFO: `/v3/projects/5365d63a880842feb79fc933412444c5/users/6625e1b1d5164bea8cb0010becb1e7d2/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:00.824321 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9662443d-454b-4652-935c-f111f3dd2839 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5365d63a880842feb79fc933412444c5, user_id=6625e1b1d5164bea8cb0010becb1e7d2, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:00.827004 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9662443d-454b-4652-935c-f111f3dd2839 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:00.839619 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1120/2249] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:21:00 2022] PUT /identity/v3/projects/5365d63a880842feb79fc933412444c5/users/6625e1b1d5164bea8cb0010becb1e7d2/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:21:01.817964 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d8f16-4635-4821-9017-56755eb9be9f None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:01.819019 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d8f16-4635-4821-9017-56755eb9be9f None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:01.822043 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d8f16-4635-4821-9017-56755eb9be9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:01.869127 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-c23d8f16-4635-4821-9017-56755eb9be9f None None] MFA Rules not processed for user `6625e1b1d5164bea8cb0010becb1e7d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:21:01.869365 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-c23d8f16-4635-4821-9017-56755eb9be9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:01.940404 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:02.003834 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1130/2250] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:21:01 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:21:03.410183 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662443d-454b-4652-935c-f111f3dd2839 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:03.507534 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662443d-454b-4652-935c-f111f3dd2839 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:03.512878 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb82d2ba-c682-4e2e-8ab8-7d0831922bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:03.513816 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82d2ba-c682-4e2e-8ab8-7d0831922bbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:03.514127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82d2ba-c682-4e2e-8ab8-7d0831922bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:03.514394 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb82d2ba-c682-4e2e-8ab8-7d0831922bbc admin admin] PATH_INFO: `/v3/users/f50635e25f7743c38cd2cf2fefff39c1` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:03.515639 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb82d2ba-c682-4e2e-8ab8-7d0831922bbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f50635e25f7743c38cd2cf2fefff39c1)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:03.517629 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb82d2ba-c682-4e2e-8ab8-7d0831922bbc admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:03.534222 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:03.586607 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-bb82d2ba-c682-4e2e-8ab8-7d0831922bbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f50635e25f7743c38cd2cf2fefff39c1'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:03.595366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-bb82d2ba-c682-4e2e-8ab8-7d0831922bbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f50635e25f7743c38cd2cf2fefff39c1'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:03.596269 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-bb82d2ba-c682-4e2e-8ab8-7d0831922bbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f50635e25f7743c38cd2cf2fefff39c1'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:03.597759 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-bb82d2ba-c682-4e2e-8ab8-7d0831922bbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f50635e25f7743c38cd2cf2fefff39c1'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:03.608671 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-bb82d2ba-c682-4e2e-8ab8-7d0831922bbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f50635e25f7743c38cd2cf2fefff39c1'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:03.627675 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1121/2251] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:21:03 2022] DELETE /identity/v3/users/f50635e25f7743c38cd2cf2fefff39c1 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:21:03.881343 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23d8f16-4635-4821-9017-56755eb9be9f None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:03.893583 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23d8f16-4635-4821-9017-56755eb9be9f None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:03.963971 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65b6199-a6c3-4a5c-a5d0-ff81616f9418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:03.965081 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65b6199-a6c3-4a5c-a5d0-ff81616f9418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:03.965410 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65b6199-a6c3-4a5c-a5d0-ff81616f9418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:03.965702 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65b6199-a6c3-4a5c-a5d0-ff81616f9418 admin admin] PATH_INFO: `/v3/projects/6370ee04d3a14bb2a59bde4e79c70db6` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:03.967021 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65b6199-a6c3-4a5c-a5d0-ff81616f9418 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6370ee04d3a14bb2a59bde4e79c70db6)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:03.969042 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65b6199-a6c3-4a5c-a5d0-ff81616f9418 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:04.019878 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a65b6199-a6c3-4a5c-a5d0-ff81616f9418 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6370ee04d3a14bb2a59bde4e79c70db6', 'request_id': 'req-a65b6199-a6c3-4a5c-a5d0-ff81616f9418'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:04.030495 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a65b6199-a6c3-4a5c-a5d0-ff81616f9418 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6370ee04d3a14bb2a59bde4e79c70db6', 'request_id': 'req-a65b6199-a6c3-4a5c-a5d0-ff81616f9418'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:04.040606 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a65b6199-a6c3-4a5c-a5d0-ff81616f9418 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6370ee04d3a14bb2a59bde4e79c70db6', 'request_id': 'req-a65b6199-a6c3-4a5c-a5d0-ff81616f9418'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:04.053094 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a65b6199-a6c3-4a5c-a5d0-ff81616f9418 admin admin] The token cache is being invalidate because project 6370ee04d3a14bb2a59bde4e79c70db6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:21:04.053390 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a65b6199-a6c3-4a5c-a5d0-ff81616f9418 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:04.054165 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a65b6199-a6c3-4a5c-a5d0-ff81616f9418 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:04.055981 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1131/2252] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:21:03 2022] DELETE /identity/v3/projects/6370ee04d3a14bb2a59bde4e79c70db6 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:21:04.060232 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb82d2ba-c682-4e2e-8ab8-7d0831922bbc admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:04.061827 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-bb82d2ba-c682-4e2e-8ab8-7d0831922bbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:04.076015 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb82d2ba-c682-4e2e-8ab8-7d0831922bbc admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:04.156800 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16064fae-fe94-417e-81d5-3123a4fdf20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:04.158282 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16064fae-fe94-417e-81d5-3123a4fdf20f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:04.158748 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16064fae-fe94-417e-81d5-3123a4fdf20f admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:04.159199 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16064fae-fe94-417e-81d5-3123a4fdf20f admin admin] PATH_INFO: `/v3/users/6625e1b1d5164bea8cb0010becb1e7d2` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:04.161417 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16064fae-fe94-417e-81d5-3123a4fdf20f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6625e1b1d5164bea8cb0010becb1e7d2)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:04.164832 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16064fae-fe94-417e-81d5-3123a4fdf20f admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:04.247814 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-16064fae-fe94-417e-81d5-3123a4fdf20f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6625e1b1d5164bea8cb0010becb1e7d2'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:04.257354 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-16064fae-fe94-417e-81d5-3123a4fdf20f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6625e1b1d5164bea8cb0010becb1e7d2'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:04.258421 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-16064fae-fe94-417e-81d5-3123a4fdf20f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6625e1b1d5164bea8cb0010becb1e7d2'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:04.259696 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-16064fae-fe94-417e-81d5-3123a4fdf20f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6625e1b1d5164bea8cb0010becb1e7d2'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:04.275497 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-16064fae-fe94-417e-81d5-3123a4fdf20f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6625e1b1d5164bea8cb0010becb1e7d2'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:04.297270 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1122/2253] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:21:04 2022] DELETE /identity/v3/users/6625e1b1d5164bea8cb0010becb1e7d2 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:21:04.499503 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65b6199-a6c3-4a5c-a5d0-ff81616f9418 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:04.501828 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a65b6199-a6c3-4a5c-a5d0-ff81616f9418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:04.518859 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65b6199-a6c3-4a5c-a5d0-ff81616f9418 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:04.585958 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b397ad-086d-4ecb-863a-7da55cb7e2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:04.586994 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b397ad-086d-4ecb-863a-7da55cb7e2f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:04.587367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b397ad-086d-4ecb-863a-7da55cb7e2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:04.587748 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b397ad-086d-4ecb-863a-7da55cb7e2f4 admin admin] PATH_INFO: `/v3/projects/5365d63a880842feb79fc933412444c5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:04.589118 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b397ad-086d-4ecb-863a-7da55cb7e2f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5365d63a880842feb79fc933412444c5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:04.591232 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b397ad-086d-4ecb-863a-7da55cb7e2f4 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:04.605494 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:04.661337 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-f8b397ad-086d-4ecb-863a-7da55cb7e2f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5365d63a880842feb79fc933412444c5', 'request_id': 'req-f8b397ad-086d-4ecb-863a-7da55cb7e2f4'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:04.668862 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-f8b397ad-086d-4ecb-863a-7da55cb7e2f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5365d63a880842feb79fc933412444c5', 'request_id': 'req-f8b397ad-086d-4ecb-863a-7da55cb7e2f4'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:04.679111 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-f8b397ad-086d-4ecb-863a-7da55cb7e2f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5365d63a880842feb79fc933412444c5', 'request_id': 'req-f8b397ad-086d-4ecb-863a-7da55cb7e2f4'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:04.692885 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-f8b397ad-086d-4ecb-863a-7da55cb7e2f4 admin admin] The token cache is being invalidate because project 5365d63a880842feb79fc933412444c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:21:04.693224 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-f8b397ad-086d-4ecb-863a-7da55cb7e2f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:04.693996 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-f8b397ad-086d-4ecb-863a-7da55cb7e2f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:04.695687 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1132/2254] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:21:04 2022] DELETE /identity/v3/projects/5365d63a880842feb79fc933412444c5 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:21:04.702387 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86c701-3350-419c-86d4-e2ae3a54a3ef None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:04.705063 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86c701-3350-419c-86d4-e2ae3a54a3ef None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:04.705063 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86c701-3350-419c-86d4-e2ae3a54a3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:04.724394 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-fa86c701-3350-419c-86d4-e2ae3a54a3ef None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:21:04.725663 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-fa86c701-3350-419c-86d4-e2ae3a54a3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:04.733831 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:04.833233 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1123/2255] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:21:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:21:04.842407 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc0c94-53ca-4957-aaf8-ea278bf21f58 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:04.842761 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc0c94-53ca-4957-aaf8-ea278bf21f58 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:04.843164 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc0c94-53ca-4957-aaf8-ea278bf21f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:04.865895 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-0fcc0c94-53ca-4957-aaf8-ea278bf21f58 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:21:04.867155 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-0fcc0c94-53ca-4957-aaf8-ea278bf21f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:04.875975 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1133/2256] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:21:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:21:04.881417 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86c701-3350-419c-86d4-e2ae3a54a3ef None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:04.898135 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86c701-3350-419c-86d4-e2ae3a54a3ef None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:04.903800 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf81cd7-256d-4b50-b9db-effc64e2dc5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:04.904770 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf81cd7-256d-4b50-b9db-effc64e2dc5f admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:04.905233 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf81cd7-256d-4b50-b9db-effc64e2dc5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:04.905679 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf81cd7-256d-4b50-b9db-effc64e2dc5f admin admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:04.906589 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf81cd7-256d-4b50-b9db-effc64e2dc5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:21:04.907069 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf81cd7-256d-4b50-b9db-effc64e2dc5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:04.908859 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf81cd7-256d-4b50-b9db-effc64e2dc5f admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:04.928149 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1124/2257] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:21:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:21:04.933656 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc0c94-53ca-4957-aaf8-ea278bf21f58 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:04.951909 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc0c94-53ca-4957-aaf8-ea278bf21f58 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:04.960582 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcfac6c-c996-4dea-b0a2-590b7ed515b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:04.961613 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcfac6c-c996-4dea-b0a2-590b7ed515b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:04.962324 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcfac6c-c996-4dea-b0a2-590b7ed515b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:04.962787 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcfac6c-c996-4dea-b0a2-590b7ed515b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:04.963505 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcfac6c-c996-4dea-b0a2-590b7ed515b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:04.965517 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcfac6c-c996-4dea-b0a2-590b7ed515b7 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:05.005023 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1134/2258] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:21:04 2022] POST /identity/v3/projects => generated 367 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:21:05.007382 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf81cd7-256d-4b50-b9db-effc64e2dc5f admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:05.049012 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf81cd7-256d-4b50-b9db-effc64e2dc5f admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:05.127032 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82a9f7c-dfdf-493b-8530-a53b15f3090c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:05.127751 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a9f7c-dfdf-493b-8530-a53b15f3090c admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:05.128034 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a9f7c-dfdf-493b-8530-a53b15f3090c admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:05.128313 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a9f7c-dfdf-493b-8530-a53b15f3090c admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:05.129006 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82a9f7c-dfdf-493b-8530-a53b15f3090c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:05.130933 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82a9f7c-dfdf-493b-8530-a53b15f3090c admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:05.154145 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1125/2259] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:21:05 2022] POST /identity/v3/users => generated 284 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:21:05.159108 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcfac6c-c996-4dea-b0a2-590b7ed515b7 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:05.172606 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcfac6c-c996-4dea-b0a2-590b7ed515b7 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:05.179048 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8daab3f-ce99-4af0-ab03-6b7c6a47dfd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:05.179763 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8daab3f-ce99-4af0-ab03-6b7c6a47dfd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:05.180046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8daab3f-ce99-4af0-ab03-6b7c6a47dfd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:05.180323 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8daab3f-ce99-4af0-ab03-6b7c6a47dfd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:05.181411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8daab3f-ce99-4af0-ab03-6b7c6a47dfd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:05.183098 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8daab3f-ce99-4af0-ab03-6b7c6a47dfd6 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:05.195858 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1135/2260] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:21:05.201127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82a9f7c-dfdf-493b-8530-a53b15f3090c admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:05.215702 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82a9f7c-dfdf-493b-8530-a53b15f3090c admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:05.221323 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2751754-d6be-4068-b69a-fd26c5115438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:05.222063 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2751754-d6be-4068-b69a-fd26c5115438 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:05.222459 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2751754-d6be-4068-b69a-fd26c5115438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:05.222800 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2751754-d6be-4068-b69a-fd26c5115438 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:05.223639 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2751754-d6be-4068-b69a-fd26c5115438 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:05.225371 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2751754-d6be-4068-b69a-fd26c5115438 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:05.239248 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1126/2261] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:21:05.245162 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8daab3f-ce99-4af0-ab03-6b7c6a47dfd6 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:05.255834 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8daab3f-ce99-4af0-ab03-6b7c6a47dfd6 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:05.261663 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb5c952-291a-4037-815d-5f77771de68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:05.263282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb5c952-291a-4037-815d-5f77771de68c admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:05.263781 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb5c952-291a-4037-815d-5f77771de68c admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:05.263890 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb5c952-291a-4037-815d-5f77771de68c admin admin] PATH_INFO: `/v3/projects/dae2a6f8fb6142d8991641feafdfff26/users/dc5794a647fa493d882a04d6ef451c42/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:05.283426 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb5c952-291a-4037-815d-5f77771de68c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dae2a6f8fb6142d8991641feafdfff26, user_id=dc5794a647fa493d882a04d6ef451c42, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:05.286341 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb5c952-291a-4037-815d-5f77771de68c admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:05.305853 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1136/2262] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:21:05 2022] PUT /identity/v3/projects/dae2a6f8fb6142d8991641feafdfff26/users/dc5794a647fa493d882a04d6ef451c42/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:21:05.324026 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3f5e2-04ff-4fb2-b7c5-f098520d82bc None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:05.324538 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3f5e2-04ff-4fb2-b7c5-f098520d82bc None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:05.325092 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3f5e2-04ff-4fb2-b7c5-f098520d82bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:05.395384 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-bfb3f5e2-04ff-4fb2-b7c5-f098520d82bc None None] MFA Rules not processed for user `dc5794a647fa493d882a04d6ef451c42`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:21:05.397618 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-bfb3f5e2-04ff-4fb2-b7c5-f098520d82bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:05.528991 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1127/2263] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:21:05 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:21:05.529213 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb5c952-291a-4037-815d-5f77771de68c admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:05.632751 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb5c952-291a-4037-815d-5f77771de68c admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:05.638199 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61079df1-af5b-4db2-b710-f52f8d48c86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:05.639110 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61079df1-af5b-4db2-b710-f52f8d48c86f admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:05.639453 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61079df1-af5b-4db2-b710-f52f8d48c86f admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:05.639791 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61079df1-af5b-4db2-b710-f52f8d48c86f admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:05.640620 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61079df1-af5b-4db2-b710-f52f8d48c86f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:05.642794 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61079df1-af5b-4db2-b710-f52f8d48c86f admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:05.670419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1137/2264] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:21:05 2022] POST /identity/v3/projects => generated 369 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:21:05.674647 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3f5e2-04ff-4fb2-b7c5-f098520d82bc None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:05.685921 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3f5e2-04ff-4fb2-b7c5-f098520d82bc None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:05.689739 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:05.748113 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab03f141-9d4a-4e31-8ff9-16f939592176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:05.748841 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03f141-9d4a-4e31-8ff9-16f939592176 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:05.749124 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03f141-9d4a-4e31-8ff9-16f939592176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:05.749402 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03f141-9d4a-4e31-8ff9-16f939592176 admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:05.750130 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab03f141-9d4a-4e31-8ff9-16f939592176 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:05.752485 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab03f141-9d4a-4e31-8ff9-16f939592176 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:05.774147 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1128/2265] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:21:05 2022] POST /identity/v3/users => generated 291 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:21:05.778881 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61079df1-af5b-4db2-b710-f52f8d48c86f admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:05.789727 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61079df1-af5b-4db2-b710-f52f8d48c86f admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:05.794809 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a076de58-14de-404b-8387-5e61ef2fbc6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:05.795474 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076de58-14de-404b-8387-5e61ef2fbc6e admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:05.795753 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076de58-14de-404b-8387-5e61ef2fbc6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:05.796021 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076de58-14de-404b-8387-5e61ef2fbc6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:05.796783 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a076de58-14de-404b-8387-5e61ef2fbc6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:05.798427 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a076de58-14de-404b-8387-5e61ef2fbc6e admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:05.810802 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1138/2266] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:21:05.816309 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab03f141-9d4a-4e31-8ff9-16f939592176 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:05.832686 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab03f141-9d4a-4e31-8ff9-16f939592176 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:05.835804 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564d918c-9a5f-4cea-ab03-4c5b5ee69286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:05.837024 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d918c-9a5f-4cea-ab03-4c5b5ee69286 admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:05.837449 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d918c-9a5f-4cea-ab03-4c5b5ee69286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:05.837784 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d918c-9a5f-4cea-ab03-4c5b5ee69286 admin admin] PATH_INFO: `/v3/projects/94c1fc66e705414ea27822abd5951d62/users/467298e5ec7a4ef89739ff762e79d25d/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:05.859956 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564d918c-9a5f-4cea-ab03-4c5b5ee69286 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94c1fc66e705414ea27822abd5951d62, user_id=467298e5ec7a4ef89739ff762e79d25d, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:05.862693 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564d918c-9a5f-4cea-ab03-4c5b5ee69286 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:05.878876 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1129/2267] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:21:05 2022] PUT /identity/v3/projects/94c1fc66e705414ea27822abd5951d62/users/467298e5ec7a4ef89739ff762e79d25d/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:21:05.885416 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a076de58-14de-404b-8387-5e61ef2fbc6e admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:05.898328 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a076de58-14de-404b-8387-5e61ef2fbc6e admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:05.905140 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:05.980668 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd72645e-ff49-40fb-9617-89293e5db6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:05.981724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72645e-ff49-40fb-9617-89293e5db6fc admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:05.981955 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72645e-ff49-40fb-9617-89293e5db6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:05.982132 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd72645e-ff49-40fb-9617-89293e5db6fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:05.982810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd72645e-ff49-40fb-9617-89293e5db6fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:05.984443 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd72645e-ff49-40fb-9617-89293e5db6fc admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:05.997451 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1139/2268] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:21:06.002455 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564d918c-9a5f-4cea-ab03-4c5b5ee69286 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:06.014289 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564d918c-9a5f-4cea-ab03-4c5b5ee69286 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:06.019578 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88af4192-49d2-459b-b103-874e2d4fae0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:06.020640 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af4192-49d2-459b-b103-874e2d4fae0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:06.021019 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af4192-49d2-459b-b103-874e2d4fae0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:06.021311 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af4192-49d2-459b-b103-874e2d4fae0e admin admin] PATH_INFO: `/v3/projects/94c1fc66e705414ea27822abd5951d62/users/467298e5ec7a4ef89739ff762e79d25d/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:06.023429 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88af4192-49d2-459b-b103-874e2d4fae0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=94c1fc66e705414ea27822abd5951d62, user_id=467298e5ec7a4ef89739ff762e79d25d, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:06.026520 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88af4192-49d2-459b-b103-874e2d4fae0e admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:06.038015 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1130/2269] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:21:06 2022] PUT /identity/v3/projects/94c1fc66e705414ea27822abd5951d62/users/467298e5ec7a4ef89739ff762e79d25d/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:21:06.049877 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a188ab-e817-44be-a9e8-cb428462669b None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:06.050217 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a188ab-e817-44be-a9e8-cb428462669b None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:06.050564 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a188ab-e817-44be-a9e8-cb428462669b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:06.079418 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-d0a188ab-e817-44be-a9e8-cb428462669b None None] MFA Rules not processed for user `467298e5ec7a4ef89739ff762e79d25d`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:21:06.080644 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d0a188ab-e817-44be-a9e8-cb428462669b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:06.154097 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1140/2270] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:21:06 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:21:06.166727 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88af4192-49d2-459b-b103-874e2d4fae0e admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:06.168660 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-88af4192-49d2-459b-b103-874e2d4fae0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:06.265706 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88af4192-49d2-459b-b103-874e2d4fae0e admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:06.272158 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921b487-cccd-4508-9bec-72da5094f7e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:06.273159 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921b487-cccd-4508-9bec-72da5094f7e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:06.273583 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921b487-cccd-4508-9bec-72da5094f7e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:06.273996 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921b487-cccd-4508-9bec-72da5094f7e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:06.286674 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b921b487-cccd-4508-9bec-72da5094f7e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:06.288113 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b921b487-cccd-4508-9bec-72da5094f7e3 service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:06.298669 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1131/2271] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:21:06 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:21:06.354660 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a188ab-e817-44be-a9e8-cb428462669b None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:06.356197 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d0a188ab-e817-44be-a9e8-cb428462669b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:06.370684 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a188ab-e817-44be-a9e8-cb428462669b None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:06.375403 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7303e1-a12a-4841-99d8-cd1d427f1b34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:06.376076 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7303e1-a12a-4841-99d8-cd1d427f1b34 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:06.376360 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7303e1-a12a-4841-99d8-cd1d427f1b34 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:06.376634 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7303e1-a12a-4841-99d8-cd1d427f1b34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:06.438046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7303e1-a12a-4841-99d8-cd1d427f1b34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:06.439629 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7303e1-a12a-4841-99d8-cd1d427f1b34 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:06.450293 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1141/2272] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:21:06 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:21:08.686255 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55b9e2-87e1-4029-a261-46b33065ef80 None None] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:08.686863 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55b9e2-87e1-4029-a261-46b33065ef80 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:08.686863 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55b9e2-87e1-4029-a261-46b33065ef80 None None] PATH_INFO: `/` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:08.688539 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1132/2273] 10.222.0.13 () {54 vars in 921 bytes} [Mon Jun 27 21:21:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:21:08.695835 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0b48e-96b6-4d36-b922-fa52eec7e729 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:08.696681 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0b48e-96b6-4d36-b922-fa52eec7e729 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:08.697398 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0b48e-96b6-4d36-b922-fa52eec7e729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:08.722482 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-afc0b48e-96b6-4d36-b922-fa52eec7e729 None None] MFA Rules not processed for user `2a4e105858324391b5142d57e34aff9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:21:08.724186 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-afc0b48e-96b6-4d36-b922-fa52eec7e729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:08.763893 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:08.797469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1142/2274] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Jun 27 21:21:08 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:21:08.811313 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f55b9e2-87e1-4029-a261-46b33065ef80 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:08.813188 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-7f55b9e2-87e1-4029-a261-46b33065ef80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:08.831833 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f55b9e2-87e1-4029-a261-46b33065ef80 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:08.837328 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a85f2de-9504-4377-85cb-f9ef5a5bfc7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:08.838204 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85f2de-9504-4377-85cb-f9ef5a5bfc7b service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:08.838351 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85f2de-9504-4377-85cb-f9ef5a5bfc7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:08.838657 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85f2de-9504-4377-85cb-f9ef5a5bfc7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:08.849221 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a85f2de-9504-4377-85cb-f9ef5a5bfc7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:08.851102 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a85f2de-9504-4377-85cb-f9ef5a5bfc7b service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:08.863126 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1133/2275] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:21:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:21:08.955452 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657ce51-01ec-4729-81e1-1a65f59d089e None None] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:08.955785 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657ce51-01ec-4729-81e1-1a65f59d089e None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:08.956139 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657ce51-01ec-4729-81e1-1a65f59d089e None None] PATH_INFO: `/` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:08.957892 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1143/2276] 10.222.0.13 () {54 vars in 921 bytes} [Mon Jun 27 21:21:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 27 21:21:08.964946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9cca9-863f-4ea5-9e4c-09b1a35a5723 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:08.965413 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9cca9-863f-4ea5-9e4c-09b1a35a5723 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:08.965691 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9cca9-863f-4ea5-9e4c-09b1a35a5723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:08.992012 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-dfc9cca9-863f-4ea5-9e4c-09b1a35a5723 None None] MFA Rules not processed for user `1366958034d540b4ba2feb421a6afabc`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:21:08.993449 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-dfc9cca9-863f-4ea5-9e4c-09b1a35a5723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:09.021817 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:09.079850 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1134/2277] 10.222.0.13 () {58 vars in 1018 bytes} [Mon Jun 27 21:21:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:21:09.093044 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c657ce51-01ec-4729-81e1-1a65f59d089e None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:09.095468 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-c657ce51-01ec-4729-81e1-1a65f59d089e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:09.112057 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c657ce51-01ec-4729-81e1-1a65f59d089e None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:09.116273 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3160cc-9316-4fec-a800-2494bd3e6473 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:09.117119 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3160cc-9316-4fec-a800-2494bd3e6473 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:09.117454 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3160cc-9316-4fec-a800-2494bd3e6473 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:09.117740 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3160cc-9316-4fec-a800-2494bd3e6473 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:09.131352 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3160cc-9316-4fec-a800-2494bd3e6473 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:09.133245 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3160cc-9316-4fec-a800-2494bd3e6473 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:09.148295 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1144/2278] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:21:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:21:09.233930 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc9cca9-863f-4ea5-9e4c-09b1a35a5723 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:09.235560 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-dfc9cca9-863f-4ea5-9e4c-09b1a35a5723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:09.334803 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc9cca9-863f-4ea5-9e4c-09b1a35a5723 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:09.341766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491d2d41-3114-4203-a18e-fe68d9c93ee9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0437c45d72cd4a38b9d48ee6f50946fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:09.342450 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d2d41-3114-4203-a18e-fe68d9c93ee9 service placement] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:09.342738 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d2d41-3114-4203-a18e-fe68d9c93ee9 service placement] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:09.343024 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d2d41-3114-4203-a18e-fe68d9c93ee9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:09.348707 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-491d2d41-3114-4203-a18e-fe68d9c93ee9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:09.363490 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491d2d41-3114-4203-a18e-fe68d9c93ee9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:09.366780 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491d2d41-3114-4203-a18e-fe68d9c93ee9 service placement] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:09.380965 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1135/2279] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:21:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:21:10.349761 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3160cc-9316-4fec-a800-2494bd3e6473 service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:10.440074 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3160cc-9316-4fec-a800-2494bd3e6473 service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:10.447529 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490b2d4f-c6fc-4783-9482-af8047360aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:10.448775 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b2d4f-c6fc-4783-9482-af8047360aca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:10.449020 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b2d4f-c6fc-4783-9482-af8047360aca admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:10.449325 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b2d4f-c6fc-4783-9482-af8047360aca admin admin] PATH_INFO: `/v3/users/dc5794a647fa493d882a04d6ef451c42` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:10.450927 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490b2d4f-c6fc-4783-9482-af8047360aca admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc5794a647fa493d882a04d6ef451c42)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:10.453115 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490b2d4f-c6fc-4783-9482-af8047360aca admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:10.547279 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-490b2d4f-c6fc-4783-9482-af8047360aca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc5794a647fa493d882a04d6ef451c42'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:10.560187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-490b2d4f-c6fc-4783-9482-af8047360aca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc5794a647fa493d882a04d6ef451c42'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:10.561117 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-490b2d4f-c6fc-4783-9482-af8047360aca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc5794a647fa493d882a04d6ef451c42'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:10.561740 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-490b2d4f-c6fc-4783-9482-af8047360aca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc5794a647fa493d882a04d6ef451c42'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:10.574663 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-490b2d4f-c6fc-4783-9482-af8047360aca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc5794a647fa493d882a04d6ef451c42'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:10.626896 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1145/2280] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:21:10 2022] DELETE /identity/v3/users/dc5794a647fa493d882a04d6ef451c42 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:21:10.635791 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3cff99-dd49-41d5-8c4d-9073d5fbdebf None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:10.636472 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3cff99-dd49-41d5-8c4d-9073d5fbdebf None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:10.637182 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3cff99-dd49-41d5-8c4d-9073d5fbdebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:10.682309 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-0a3cff99-dd49-41d5-8c4d-9073d5fbdebf None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:21:10.683646 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-0a3cff99-dd49-41d5-8c4d-9073d5fbdebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:10.800373 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1136/2281] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:21:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:21:10.815307 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490b2d4f-c6fc-4783-9482-af8047360aca admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:10.816550 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-490b2d4f-c6fc-4783-9482-af8047360aca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:10.954110 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:10.972024 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490b2d4f-c6fc-4783-9482-af8047360aca admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:10.976261 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2171fa97-bd3a-4bed-b0c1-d17094d93f55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:10.977045 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171fa97-bd3a-4bed-b0c1-d17094d93f55 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:10.978756 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171fa97-bd3a-4bed-b0c1-d17094d93f55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:10.979198 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171fa97-bd3a-4bed-b0c1-d17094d93f55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:10.996478 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2171fa97-bd3a-4bed-b0c1-d17094d93f55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:10.998476 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2171fa97-bd3a-4bed-b0c1-d17094d93f55 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:11.013133 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1146/2282] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:21:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:21:11.790783 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3cff99-dd49-41d5-8c4d-9073d5fbdebf None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:11.793066 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-0a3cff99-dd49-41d5-8c4d-9073d5fbdebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:11.813478 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3cff99-dd49-41d5-8c4d-9073d5fbdebf None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:11.820015 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab66f32-dafa-485f-a93f-44ea0e26b274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:11.821499 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab66f32-dafa-485f-a93f-44ea0e26b274 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:11.821982 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab66f32-dafa-485f-a93f-44ea0e26b274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:11.822826 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab66f32-dafa-485f-a93f-44ea0e26b274 admin admin] PATH_INFO: `/v3/projects/dae2a6f8fb6142d8991641feafdfff26` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:11.824466 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab66f32-dafa-485f-a93f-44ea0e26b274 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dae2a6f8fb6142d8991641feafdfff26)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:11.826918 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab66f32-dafa-485f-a93f-44ea0e26b274 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:11.905241 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:11.907932 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-7ab66f32-dafa-485f-a93f-44ea0e26b274 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dae2a6f8fb6142d8991641feafdfff26', 'request_id': 'req-7ab66f32-dafa-485f-a93f-44ea0e26b274'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:11.919985 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-7ab66f32-dafa-485f-a93f-44ea0e26b274 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dae2a6f8fb6142d8991641feafdfff26', 'request_id': 'req-7ab66f32-dafa-485f-a93f-44ea0e26b274'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:11.949476 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-7ab66f32-dafa-485f-a93f-44ea0e26b274 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dae2a6f8fb6142d8991641feafdfff26', 'request_id': 'req-7ab66f32-dafa-485f-a93f-44ea0e26b274'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:11.971081 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-7ab66f32-dafa-485f-a93f-44ea0e26b274 admin admin] The token cache is being invalidate because project dae2a6f8fb6142d8991641feafdfff26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:21:11.971666 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-7ab66f32-dafa-485f-a93f-44ea0e26b274 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:11.972681 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-7ab66f32-dafa-485f-a93f-44ea0e26b274 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:11.974958 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1137/2283] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:21:11 2022] DELETE /identity/v3/projects/dae2a6f8fb6142d8991641feafdfff26 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:21:11.979887 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2171fa97-bd3a-4bed-b0c1-d17094d93f55 service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:11.992577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2171fa97-bd3a-4bed-b0c1-d17094d93f55 service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:12.116360 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4996a9-f6ad-4b2c-83a9-bafb6e32a5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:12.119996 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4996a9-f6ad-4b2c-83a9-bafb6e32a5f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:12.120502 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4996a9-f6ad-4b2c-83a9-bafb6e32a5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:12.121268 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4996a9-f6ad-4b2c-83a9-bafb6e32a5f9 admin admin] PATH_INFO: `/v3/users/467298e5ec7a4ef89739ff762e79d25d` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:12.123492 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4996a9-f6ad-4b2c-83a9-bafb6e32a5f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=467298e5ec7a4ef89739ff762e79d25d)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:12.126251 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4996a9-f6ad-4b2c-83a9-bafb6e32a5f9 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:12.200861 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-2b4996a9-f6ad-4b2c-83a9-bafb6e32a5f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '467298e5ec7a4ef89739ff762e79d25d'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:12.221231 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-2b4996a9-f6ad-4b2c-83a9-bafb6e32a5f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '467298e5ec7a4ef89739ff762e79d25d'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:12.222461 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-2b4996a9-f6ad-4b2c-83a9-bafb6e32a5f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '467298e5ec7a4ef89739ff762e79d25d'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:12.225961 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-2b4996a9-f6ad-4b2c-83a9-bafb6e32a5f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '467298e5ec7a4ef89739ff762e79d25d'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:12.238121 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-2b4996a9-f6ad-4b2c-83a9-bafb6e32a5f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '467298e5ec7a4ef89739ff762e79d25d'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:12.263885 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1147/2284] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:21:11 2022] DELETE /identity/v3/users/467298e5ec7a4ef89739ff762e79d25d => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:21:12.716114 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab66f32-dafa-485f-a93f-44ea0e26b274 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:12.717894 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-7ab66f32-dafa-485f-a93f-44ea0e26b274 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:12.735529 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab66f32-dafa-485f-a93f-44ea0e26b274 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:12.833442 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d9bcc1-6d50-44ac-be90-a26b6b6f0d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:12.835333 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9bcc1-6d50-44ac-be90-a26b6b6f0d1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:12.835833 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9bcc1-6d50-44ac-be90-a26b6b6f0d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:12.836345 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d9bcc1-6d50-44ac-be90-a26b6b6f0d1c admin admin] PATH_INFO: `/v3/projects/94c1fc66e705414ea27822abd5951d62` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:12.838724 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d9bcc1-6d50-44ac-be90-a26b6b6f0d1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=94c1fc66e705414ea27822abd5951d62)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:12.842684 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d9bcc1-6d50-44ac-be90-a26b6b6f0d1c admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:12.915671 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-02d9bcc1-6d50-44ac-be90-a26b6b6f0d1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94c1fc66e705414ea27822abd5951d62', 'request_id': 'req-02d9bcc1-6d50-44ac-be90-a26b6b6f0d1c'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:12.922889 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-02d9bcc1-6d50-44ac-be90-a26b6b6f0d1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94c1fc66e705414ea27822abd5951d62', 'request_id': 'req-02d9bcc1-6d50-44ac-be90-a26b6b6f0d1c'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:12.933348 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-02d9bcc1-6d50-44ac-be90-a26b6b6f0d1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94c1fc66e705414ea27822abd5951d62', 'request_id': 'req-02d9bcc1-6d50-44ac-be90-a26b6b6f0d1c'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:12.945679 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-02d9bcc1-6d50-44ac-be90-a26b6b6f0d1c admin admin] The token cache is being invalidate because project 94c1fc66e705414ea27822abd5951d62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:21:12.946128 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-02d9bcc1-6d50-44ac-be90-a26b6b6f0d1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:12.946843 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-02d9bcc1-6d50-44ac-be90-a26b6b6f0d1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:12.948588 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1138/2285] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:21:12 2022] DELETE /identity/v3/projects/94c1fc66e705414ea27822abd5951d62 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:21:12.954872 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165857d7-5fef-4688-a769-24e1cfd886ca None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:12.955406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165857d7-5fef-4688-a769-24e1cfd886ca None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:12.955725 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165857d7-5fef-4688-a769-24e1cfd886ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:12.978435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-165857d7-5fef-4688-a769-24e1cfd886ca None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:21:12.979656 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-165857d7-5fef-4688-a769-24e1cfd886ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:13.065654 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1148/2286] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:21:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:21:13.074363 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70faeb03-38b7-4d54-92a0-e8603990685d None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:13.074682 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70faeb03-38b7-4d54-92a0-e8603990685d None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:13.074938 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70faeb03-38b7-4d54-92a0-e8603990685d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:13.097291 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-70faeb03-38b7-4d54-92a0-e8603990685d None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:21:13.098863 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-70faeb03-38b7-4d54-92a0-e8603990685d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:13.109365 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1139/2287] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:21:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:21:13.114375 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165857d7-5fef-4688-a769-24e1cfd886ca None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:13.126083 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165857d7-5fef-4688-a769-24e1cfd886ca None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:13.131764 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ac694-d6b5-4893-b602-adda943d359f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:13.132508 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ac694-d6b5-4893-b602-adda943d359f admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:13.132784 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ac694-d6b5-4893-b602-adda943d359f admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:13.133085 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606ac694-d6b5-4893-b602-adda943d359f admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:13.133989 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606ac694-d6b5-4893-b602-adda943d359f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:21:13.134453 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606ac694-d6b5-4893-b602-adda943d359f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:13.136249 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606ac694-d6b5-4893-b602-adda943d359f admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:13.138227 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:13.151388 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1149/2288] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:21:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:21:13.157497 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70faeb03-38b7-4d54-92a0-e8603990685d None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:13.168441 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70faeb03-38b7-4d54-92a0-e8603990685d None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:13.174030 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df70ae8c-384f-4c09-bbd4-6b6f7b868f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:13.174769 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df70ae8c-384f-4c09-bbd4-6b6f7b868f58 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:13.175060 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df70ae8c-384f-4c09-bbd4-6b6f7b868f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:13.175337 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df70ae8c-384f-4c09-bbd4-6b6f7b868f58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:13.176088 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df70ae8c-384f-4c09-bbd4-6b6f7b868f58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:13.178196 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df70ae8c-384f-4c09-bbd4-6b6f7b868f58 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:13.209491 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1140/2289] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:21:13 2022] POST /identity/v3/projects => generated 409 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:21:13.214961 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ac694-d6b5-4893-b602-adda943d359f admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:13.228342 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606ac694-d6b5-4893-b602-adda943d359f admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:13.310261 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0972ff62-72c4-49fb-a869-afe5b80cef62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:13.310979 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972ff62-72c4-49fb-a869-afe5b80cef62 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:13.311262 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972ff62-72c4-49fb-a869-afe5b80cef62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:13.311536 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972ff62-72c4-49fb-a869-afe5b80cef62 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:13.312256 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0972ff62-72c4-49fb-a869-afe5b80cef62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:13.314352 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0972ff62-72c4-49fb-a869-afe5b80cef62 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:13.342799 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1150/2290] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:21:13 2022] POST /identity/v3/users => generated 305 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:21:13.347560 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df70ae8c-384f-4c09-bbd4-6b6f7b868f58 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:13.360414 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df70ae8c-384f-4c09-bbd4-6b6f7b868f58 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:13.366568 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91be9554-fc97-4343-b12b-49956d886674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:13.367316 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91be9554-fc97-4343-b12b-49956d886674 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:13.367664 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91be9554-fc97-4343-b12b-49956d886674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:13.367926 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91be9554-fc97-4343-b12b-49956d886674 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:13.368959 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91be9554-fc97-4343-b12b-49956d886674 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:13.371137 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91be9554-fc97-4343-b12b-49956d886674 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:13.384192 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1141/2291] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:21:13.388694 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0972ff62-72c4-49fb-a869-afe5b80cef62 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:13.400064 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0972ff62-72c4-49fb-a869-afe5b80cef62 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:13.405242 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fb93a8-418f-4c79-ba18-90fbe43cf04a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:13.405987 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb93a8-418f-4c79-ba18-90fbe43cf04a admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:13.406273 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb93a8-418f-4c79-ba18-90fbe43cf04a admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:13.406559 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb93a8-418f-4c79-ba18-90fbe43cf04a admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:13.407419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fb93a8-418f-4c79-ba18-90fbe43cf04a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:13.409378 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fb93a8-418f-4c79-ba18-90fbe43cf04a admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:13.421729 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1151/2292] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:21:13.427123 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91be9554-fc97-4343-b12b-49956d886674 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:13.439618 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91be9554-fc97-4343-b12b-49956d886674 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:13.445029 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554f404e-b85c-4572-90eb-db9ed98849fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:13.446167 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f404e-b85c-4572-90eb-db9ed98849fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:13.446457 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f404e-b85c-4572-90eb-db9ed98849fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:13.446799 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f404e-b85c-4572-90eb-db9ed98849fd admin admin] PATH_INFO: `/v3/projects/e69a9a5ec2154abc9f23520366b85c24/users/0664dff5f00a4fb5bf571574f5992b81/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:13.464892 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554f404e-b85c-4572-90eb-db9ed98849fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e69a9a5ec2154abc9f23520366b85c24, user_id=0664dff5f00a4fb5bf571574f5992b81, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:13.467767 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554f404e-b85c-4572-90eb-db9ed98849fd admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:13.478332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:13.484492 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1142/2293] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:21:13 2022] PUT /identity/v3/projects/e69a9a5ec2154abc9f23520366b85c24/users/0664dff5f00a4fb5bf571574f5992b81/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:21:13.497646 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f18ee-2303-437f-b781-57dad6a92632 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:13.497936 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f18ee-2303-437f-b781-57dad6a92632 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:13.498220 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f18ee-2303-437f-b781-57dad6a92632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:13.535387 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-2d9f18ee-2303-437f-b781-57dad6a92632 None None] MFA Rules not processed for user `0664dff5f00a4fb5bf571574f5992b81`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:21:13.536745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-2d9f18ee-2303-437f-b781-57dad6a92632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:13.611396 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1152/2294] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:21:13 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:21:13.616591 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554f404e-b85c-4572-90eb-db9ed98849fd admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:13.686076 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554f404e-b85c-4572-90eb-db9ed98849fd admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:13.691156 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b91e5-e6cc-4204-8a83-b4753b9bf282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:13.691907 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b91e5-e6cc-4204-8a83-b4753b9bf282 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:13.692197 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b91e5-e6cc-4204-8a83-b4753b9bf282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:13.692451 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b91e5-e6cc-4204-8a83-b4753b9bf282 admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:13.693257 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b91e5-e6cc-4204-8a83-b4753b9bf282 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:13.695252 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b91e5-e6cc-4204-8a83-b4753b9bf282 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:13.731693 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1143/2295] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:21:13 2022] POST /identity/v3/projects => generated 409 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:21:13.733051 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f18ee-2303-437f-b781-57dad6a92632 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:13.746246 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f18ee-2303-437f-b781-57dad6a92632 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:13.833899 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d161e7-d7ae-422c-9e44-2ba8d48d40b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:13.834934 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d161e7-d7ae-422c-9e44-2ba8d48d40b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:13.835211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d161e7-d7ae-422c-9e44-2ba8d48d40b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:13.835503 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d161e7-d7ae-422c-9e44-2ba8d48d40b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:13.836247 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d161e7-d7ae-422c-9e44-2ba8d48d40b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:13.838403 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d161e7-d7ae-422c-9e44-2ba8d48d40b9 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:13.847505 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:13.863739 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1153/2296] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:21:13 2022] POST /identity/v3/users => generated 311 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:21:13.868249 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b91e5-e6cc-4204-8a83-b4753b9bf282 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:13.879923 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b91e5-e6cc-4204-8a83-b4753b9bf282 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:13.885294 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d2978-7ba3-40c6-a00e-e6763a3dfdcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:13.886038 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d2978-7ba3-40c6-a00e-e6763a3dfdcf admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:13.886333 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d2978-7ba3-40c6-a00e-e6763a3dfdcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:13.886619 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d2978-7ba3-40c6-a00e-e6763a3dfdcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:13.887441 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98d2978-7ba3-40c6-a00e-e6763a3dfdcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:13.889209 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98d2978-7ba3-40c6-a00e-e6763a3dfdcf admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:13.902497 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1144/2297] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:21:13.907398 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d161e7-d7ae-422c-9e44-2ba8d48d40b9 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:13.918527 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d161e7-d7ae-422c-9e44-2ba8d48d40b9 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:13.923649 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2183ab7-0532-48b0-b6ed-aa39b2e54023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:13.924571 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2183ab7-0532-48b0-b6ed-aa39b2e54023 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:13.924877 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2183ab7-0532-48b0-b6ed-aa39b2e54023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:13.925195 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2183ab7-0532-48b0-b6ed-aa39b2e54023 admin admin] PATH_INFO: `/v3/projects/e98ef1ed13b94559a482a209ad69bdc4/users/9af8d0c056274874a3615f2c58848899/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:13.942434 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2183ab7-0532-48b0-b6ed-aa39b2e54023 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e98ef1ed13b94559a482a209ad69bdc4, user_id=9af8d0c056274874a3615f2c58848899, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:13.945366 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2183ab7-0532-48b0-b6ed-aa39b2e54023 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:13.962247 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1154/2298] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:21:13 2022] PUT /identity/v3/projects/e98ef1ed13b94559a482a209ad69bdc4/users/9af8d0c056274874a3615f2c58848899/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:21:13.967156 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d2978-7ba3-40c6-a00e-e6763a3dfdcf admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:13.978940 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d2978-7ba3-40c6-a00e-e6763a3dfdcf admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:14.044153 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64098d1f-d663-49e8-8c08-5d1a98f33ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:14.044872 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64098d1f-d663-49e8-8c08-5d1a98f33ccf admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:14.045185 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64098d1f-d663-49e8-8c08-5d1a98f33ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:14.045467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64098d1f-d663-49e8-8c08-5d1a98f33ccf admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:14.046238 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64098d1f-d663-49e8-8c08-5d1a98f33ccf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:14.047910 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64098d1f-d663-49e8-8c08-5d1a98f33ccf admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:14.059351 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1145/2299] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:21:14.064133 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2183ab7-0532-48b0-b6ed-aa39b2e54023 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:14.076074 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2183ab7-0532-48b0-b6ed-aa39b2e54023 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:14.081935 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f9410-6721-4119-b030-dd4b05e5b7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:14.082924 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f9410-6721-4119-b030-dd4b05e5b7ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:14.083210 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f9410-6721-4119-b030-dd4b05e5b7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:14.083488 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1f9410-6721-4119-b030-dd4b05e5b7ce admin admin] PATH_INFO: `/v3/projects/e98ef1ed13b94559a482a209ad69bdc4/users/9af8d0c056274874a3615f2c58848899/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:14.086144 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1f9410-6721-4119-b030-dd4b05e5b7ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=e98ef1ed13b94559a482a209ad69bdc4, user_id=9af8d0c056274874a3615f2c58848899, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:14.089571 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1f9410-6721-4119-b030-dd4b05e5b7ce admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:14.101592 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1155/2300] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:21:14 2022] PUT /identity/v3/projects/e98ef1ed13b94559a482a209ad69bdc4/users/9af8d0c056274874a3615f2c58848899/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:21:14.117406 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154cf95-1bef-40e0-b209-f6b179b5b042 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:14.117653 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154cf95-1bef-40e0-b209-f6b179b5b042 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:14.118194 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154cf95-1bef-40e0-b209-f6b179b5b042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:14.150881 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-1154cf95-1bef-40e0-b209-f6b179b5b042 None None] MFA Rules not processed for user `9af8d0c056274874a3615f2c58848899`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:21:14.152277 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-1154cf95-1bef-40e0-b209-f6b179b5b042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:14.224341 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:14.239942 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1146/2301] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:21:14 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:21:14.252224 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f9410-6721-4119-b030-dd4b05e5b7ce admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:14.254514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-9f1f9410-6721-4119-b030-dd4b05e5b7ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:14.317670 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1f9410-6721-4119-b030-dd4b05e5b7ce admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:14.321530 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8002c0-5cc0-4fa7-ae7a-a772a82f497a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:14.322194 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8002c0-5cc0-4fa7-ae7a-a772a82f497a service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:14.322477 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8002c0-5cc0-4fa7-ae7a-a772a82f497a service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:14.322747 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8002c0-5cc0-4fa7-ae7a-a772a82f497a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:14.333141 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8002c0-5cc0-4fa7-ae7a-a772a82f497a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:14.334978 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8002c0-5cc0-4fa7-ae7a-a772a82f497a service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:14.345659 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1156/2302] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:21:14 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:21:14.541734 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154cf95-1bef-40e0-b209-f6b179b5b042 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:14.552370 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154cf95-1bef-40e0-b209-f6b179b5b042 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:14.556429 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8086efc-60ea-45ec-8165-0a9575a403e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:14.557587 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8086efc-60ea-45ec-8165-0a9575a403e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:14.557869 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8086efc-60ea-45ec-8165-0a9575a403e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:14.558142 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8086efc-60ea-45ec-8165-0a9575a403e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:14.635721 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8086efc-60ea-45ec-8165-0a9575a403e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:14.637316 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8086efc-60ea-45ec-8165-0a9575a403e6 service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:14.647836 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1147/2303] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:21:14 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:21:16.950374 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8002c0-5cc0-4fa7-ae7a-a772a82f497a service cinder] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:16.952578 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-1d8002c0-5cc0-4fa7-ae7a-a772a82f497a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:17.049961 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8002c0-5cc0-4fa7-ae7a-a772a82f497a service cinder] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:17.055753 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e641961d-ac83-4216-837b-f0230d0d2388 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:17.056517 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641961d-ac83-4216-837b-f0230d0d2388 service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:17.056801 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641961d-ac83-4216-837b-f0230d0d2388 service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:17.057118 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641961d-ac83-4216-837b-f0230d0d2388 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:17.059116 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-e641961d-ac83-4216-837b-f0230d0d2388 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:17.118530 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:17.122917 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e641961d-ac83-4216-837b-f0230d0d2388 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:17.125206 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e641961d-ac83-4216-837b-f0230d0d2388 service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:17.135840 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1157/2304] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:21:16 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:21:19.239844 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8086efc-60ea-45ec-8165-0a9575a403e6 service cinder] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:19.255785 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8086efc-60ea-45ec-8165-0a9575a403e6 service cinder] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:19.262707 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa58ae-8f6a-48bc-a870-777b6372712d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:19.263407 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa58ae-8f6a-48bc-a870-777b6372712d service nova] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:19.263691 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa58ae-8f6a-48bc-a870-777b6372712d service nova] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:19.263968 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa58ae-8f6a-48bc-a870-777b6372712d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:19.265905 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-55fa58ae-8f6a-48bc-a870-777b6372712d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:19.335291 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fa58ae-8f6a-48bc-a870-777b6372712d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:19.336979 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fa58ae-8f6a-48bc-a870-777b6372712d service nova] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:19.347630 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1148/2305] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:21:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:21:21.014088 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456cbd10-fc23-49a6-9809-9ee1c765fea9 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:21.015473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456cbd10-fc23-49a6-9809-9ee1c765fea9 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:21.016119 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456cbd10-fc23-49a6-9809-9ee1c765fea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:21.061828 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-456cbd10-fc23-49a6-9809-9ee1c765fea9 None None] MFA Rules not processed for user `f67110d591a94537a3df6a2d835370f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:21:21.063927 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-456cbd10-fc23-49a6-9809-9ee1c765fea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:21.157127 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1158/2306] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Jun 27 21:21:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:21:21.212204 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa58ae-8f6a-48bc-a870-777b6372712d service nova] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:21.213785 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-55fa58ae-8f6a-48bc-a870-777b6372712d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:21.347021 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa58ae-8f6a-48bc-a870-777b6372712d service nova] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:21.356180 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90784a99-dd21-4060-bea2-6fa386eadf70 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bed6b231ee647d8a350da055d8e59d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:21.357408 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90784a99-dd21-4060-bea2-6fa386eadf70 service swift] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:21.357926 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90784a99-dd21-4060-bea2-6fa386eadf70 service swift] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:21.358454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90784a99-dd21-4060-bea2-6fa386eadf70 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:21.375039 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90784a99-dd21-4060-bea2-6fa386eadf70 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:21.378291 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90784a99-dd21-4060-bea2-6fa386eadf70 service swift] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:21.395759 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1149/2307] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Jun 27 21:21:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 21:21:23.438712 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456cbd10-fc23-49a6-9809-9ee1c765fea9 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:23.562720 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456cbd10-fc23-49a6-9809-9ee1c765fea9 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:23.568016 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1006434-0275-4411-a081-1f42fd571920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:23.569438 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1006434-0275-4411-a081-1f42fd571920 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:23.569725 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1006434-0275-4411-a081-1f42fd571920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:23.569999 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1006434-0275-4411-a081-1f42fd571920 admin admin] PATH_INFO: `/v3/users/0664dff5f00a4fb5bf571574f5992b81` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:23.571405 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1006434-0275-4411-a081-1f42fd571920 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0664dff5f00a4fb5bf571574f5992b81)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:23.573622 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1006434-0275-4411-a081-1f42fd571920 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:23.664080 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a1006434-0275-4411-a081-1f42fd571920 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0664dff5f00a4fb5bf571574f5992b81'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:23.680247 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a1006434-0275-4411-a081-1f42fd571920 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0664dff5f00a4fb5bf571574f5992b81'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:23.681019 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a1006434-0275-4411-a081-1f42fd571920 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0664dff5f00a4fb5bf571574f5992b81'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:23.681588 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a1006434-0275-4411-a081-1f42fd571920 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0664dff5f00a4fb5bf571574f5992b81'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:23.703280 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a1006434-0275-4411-a081-1f42fd571920 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0664dff5f00a4fb5bf571574f5992b81'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:23.735039 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1159/2308] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:21:23 2022] DELETE /identity/v3/users/0664dff5f00a4fb5bf571574f5992b81 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:21:23.739999 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ce225-e851-48cb-8bab-d5839fbe77dd None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:23.739999 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ce225-e851-48cb-8bab-d5839fbe77dd None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:23.739999 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ce225-e851-48cb-8bab-d5839fbe77dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:23.764362 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-7f7ce225-e851-48cb-8bab-d5839fbe77dd None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:21:23.765714 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-7f7ce225-e851-48cb-8bab-d5839fbe77dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:23.782930 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:23.887680 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1150/2309] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:21:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:21:23.916949 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1006434-0275-4411-a081-1f42fd571920 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:23.918511 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a1006434-0275-4411-a081-1f42fd571920 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:23.960872 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:24.035886 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1006434-0275-4411-a081-1f42fd571920 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:24.040389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd471e1-a753-4e2b-a3fc-3a0772f1a2c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:24.041117 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd471e1-a753-4e2b-a3fc-3a0772f1a2c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:24.041388 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd471e1-a753-4e2b-a3fc-3a0772f1a2c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:24.041671 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd471e1-a753-4e2b-a3fc-3a0772f1a2c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:24.059576 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd471e1-a753-4e2b-a3fc-3a0772f1a2c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:24.059576 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd471e1-a753-4e2b-a3fc-3a0772f1a2c3 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:24.077728 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1160/2310] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:21:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:21:24.531160 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7ce225-e851-48cb-8bab-d5839fbe77dd None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:24.532633 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-7f7ce225-e851-48cb-8bab-d5839fbe77dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:24.550751 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7ce225-e851-48cb-8bab-d5839fbe77dd None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:24.577538 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c989d7a3-e301-4133-bf33-5059a19e2e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:24.579095 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989d7a3-e301-4133-bf33-5059a19e2e1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:24.579559 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989d7a3-e301-4133-bf33-5059a19e2e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:24.580078 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989d7a3-e301-4133-bf33-5059a19e2e1e admin admin] PATH_INFO: `/v3/projects/e69a9a5ec2154abc9f23520366b85c24` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:24.588383 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c989d7a3-e301-4133-bf33-5059a19e2e1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e69a9a5ec2154abc9f23520366b85c24)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:24.595785 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c989d7a3-e301-4133-bf33-5059a19e2e1e admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:24.668813 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c989d7a3-e301-4133-bf33-5059a19e2e1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e69a9a5ec2154abc9f23520366b85c24', 'request_id': 'req-c989d7a3-e301-4133-bf33-5059a19e2e1e'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:24.676820 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c989d7a3-e301-4133-bf33-5059a19e2e1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e69a9a5ec2154abc9f23520366b85c24', 'request_id': 'req-c989d7a3-e301-4133-bf33-5059a19e2e1e'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:24.688593 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c989d7a3-e301-4133-bf33-5059a19e2e1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e69a9a5ec2154abc9f23520366b85c24', 'request_id': 'req-c989d7a3-e301-4133-bf33-5059a19e2e1e'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:24.702570 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c989d7a3-e301-4133-bf33-5059a19e2e1e admin admin] The token cache is being invalidate because project e69a9a5ec2154abc9f23520366b85c24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:21:24.703163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c989d7a3-e301-4133-bf33-5059a19e2e1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:24.703974 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c989d7a3-e301-4133-bf33-5059a19e2e1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:24.706941 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1151/2311] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:21:24 2022] DELETE /identity/v3/projects/e69a9a5ec2154abc9f23520366b85c24 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:21:24.711086 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd471e1-a753-4e2b-a3fc-3a0772f1a2c3 service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:24.722516 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd471e1-a753-4e2b-a3fc-3a0772f1a2c3 service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:24.807080 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a1182-e74d-4c31-84a1-c4384ef3c814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:24.807980 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a1182-e74d-4c31-84a1-c4384ef3c814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:24.808284 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a1182-e74d-4c31-84a1-c4384ef3c814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:24.808524 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a1182-e74d-4c31-84a1-c4384ef3c814 admin admin] PATH_INFO: `/v3/users/9af8d0c056274874a3615f2c58848899` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:24.809838 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8a1182-e74d-4c31-84a1-c4384ef3c814 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9af8d0c056274874a3615f2c58848899)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:24.811881 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8a1182-e74d-4c31-84a1-c4384ef3c814 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:24.908669 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-2b8a1182-e74d-4c31-84a1-c4384ef3c814 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9af8d0c056274874a3615f2c58848899'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:24.922895 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-2b8a1182-e74d-4c31-84a1-c4384ef3c814 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9af8d0c056274874a3615f2c58848899'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:24.935105 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-2b8a1182-e74d-4c31-84a1-c4384ef3c814 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9af8d0c056274874a3615f2c58848899'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:24.936324 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-2b8a1182-e74d-4c31-84a1-c4384ef3c814 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9af8d0c056274874a3615f2c58848899'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:24.952685 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-2b8a1182-e74d-4c31-84a1-c4384ef3c814 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9af8d0c056274874a3615f2c58848899'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:24.990037 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1161/2312] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:21:24 2022] DELETE /identity/v3/users/9af8d0c056274874a3615f2c58848899 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:21:25.429706 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c989d7a3-e301-4133-bf33-5059a19e2e1e admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:25.431246 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-c989d7a3-e301-4133-bf33-5059a19e2e1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:25.461435 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c989d7a3-e301-4133-bf33-5059a19e2e1e admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:25.544979 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb284da-b38e-4ff9-bc4a-feaff5d0f30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:25.545970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb284da-b38e-4ff9-bc4a-feaff5d0f30f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:25.546321 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb284da-b38e-4ff9-bc4a-feaff5d0f30f admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:25.546714 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb284da-b38e-4ff9-bc4a-feaff5d0f30f admin admin] PATH_INFO: `/v3/projects/e98ef1ed13b94559a482a209ad69bdc4` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:25.547983 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb284da-b38e-4ff9-bc4a-feaff5d0f30f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e98ef1ed13b94559a482a209ad69bdc4)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:25.549998 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb284da-b38e-4ff9-bc4a-feaff5d0f30f admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:25.613495 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:25.628582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-feb284da-b38e-4ff9-bc4a-feaff5d0f30f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e98ef1ed13b94559a482a209ad69bdc4', 'request_id': 'req-feb284da-b38e-4ff9-bc4a-feaff5d0f30f'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:25.638223 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-feb284da-b38e-4ff9-bc4a-feaff5d0f30f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e98ef1ed13b94559a482a209ad69bdc4', 'request_id': 'req-feb284da-b38e-4ff9-bc4a-feaff5d0f30f'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:25.651248 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-feb284da-b38e-4ff9-bc4a-feaff5d0f30f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e98ef1ed13b94559a482a209ad69bdc4', 'request_id': 'req-feb284da-b38e-4ff9-bc4a-feaff5d0f30f'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:25.674872 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-feb284da-b38e-4ff9-bc4a-feaff5d0f30f admin admin] The token cache is being invalidate because project e98ef1ed13b94559a482a209ad69bdc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:21:25.675152 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-feb284da-b38e-4ff9-bc4a-feaff5d0f30f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:25.675852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-feb284da-b38e-4ff9-bc4a-feaff5d0f30f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:25.678647 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1152/2313] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:21:25 2022] DELETE /identity/v3/projects/e98ef1ed13b94559a482a209ad69bdc4 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:21:25.685701 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9300cb2-91dd-4640-a62f-975543f34523 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:25.686092 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9300cb2-91dd-4640-a62f-975543f34523 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:25.686557 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9300cb2-91dd-4640-a62f-975543f34523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:25.716270 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-c9300cb2-91dd-4640-a62f-975543f34523 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:21:25.719463 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-c9300cb2-91dd-4640-a62f-975543f34523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:25.795874 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:25.829954 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1162/2314] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:21:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:21:25.840076 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c2d1b-85a6-49a7-9506-5e7de140b360 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:25.841122 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c2d1b-85a6-49a7-9506-5e7de140b360 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:25.842177 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c2d1b-85a6-49a7-9506-5e7de140b360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:25.884564 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-6f1c2d1b-85a6-49a7-9506-5e7de140b360 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:21:25.888100 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-6f1c2d1b-85a6-49a7-9506-5e7de140b360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:25.910144 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1153/2315] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:21:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:21:25.914803 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9300cb2-91dd-4640-a62f-975543f34523 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:25.934830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9300cb2-91dd-4640-a62f-975543f34523 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:25.943328 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb896f-37aa-4571-8fe0-0cc2ea7f26ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:25.945516 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb896f-37aa-4571-8fe0-0cc2ea7f26ea admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:25.946676 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb896f-37aa-4571-8fe0-0cc2ea7f26ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:25.947219 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb896f-37aa-4571-8fe0-0cc2ea7f26ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:25.948409 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bb896f-37aa-4571-8fe0-0cc2ea7f26ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:21:25.949078 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bb896f-37aa-4571-8fe0-0cc2ea7f26ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:25.951352 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bb896f-37aa-4571-8fe0-0cc2ea7f26ea admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:25.978175 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1163/2316] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:21:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:21:25.987855 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c2d1b-85a6-49a7-9506-5e7de140b360 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:26.020852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c2d1b-85a6-49a7-9506-5e7de140b360 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:26.039338 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477aeff8-4273-418a-87a7-1ff52e04c426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:26.040077 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477aeff8-4273-418a-87a7-1ff52e04c426 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:26.040356 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477aeff8-4273-418a-87a7-1ff52e04c426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:26.040630 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477aeff8-4273-418a-87a7-1ff52e04c426 admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:26.042241 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477aeff8-4273-418a-87a7-1ff52e04c426 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:26.044632 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477aeff8-4273-418a-87a7-1ff52e04c426 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:26.106247 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1154/2317] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:21:25 2022] POST /identity/v3/projects => generated 385 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:21:26.110027 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb896f-37aa-4571-8fe0-0cc2ea7f26ea admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:26.127831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb896f-37aa-4571-8fe0-0cc2ea7f26ea admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:26.218352 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477aeff8-4273-418a-87a7-1ff52e04c426 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:26.220783 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-477aeff8-4273-418a-87a7-1ff52e04c426 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:26.227186 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905729cb-319a-4d3c-b551-4fc83a498b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:26.228916 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905729cb-319a-4d3c-b551-4fc83a498b1e admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:26.229386 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905729cb-319a-4d3c-b551-4fc83a498b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:26.230440 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905729cb-319a-4d3c-b551-4fc83a498b1e admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:26.231604 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905729cb-319a-4d3c-b551-4fc83a498b1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:26.234487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905729cb-319a-4d3c-b551-4fc83a498b1e admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:26.272530 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1164/2318] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:21:26 2022] POST /identity/v3/users => generated 293 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:21:26.281251 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905729cb-319a-4d3c-b551-4fc83a498b1e admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:26.298431 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905729cb-319a-4d3c-b551-4fc83a498b1e admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:26.310858 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffb905d-2a21-41ac-be3e-fab1204b0320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:26.311585 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffb905d-2a21-41ac-be3e-fab1204b0320 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:26.313172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffb905d-2a21-41ac-be3e-fab1204b0320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:26.313690 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffb905d-2a21-41ac-be3e-fab1204b0320 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:26.314972 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffb905d-2a21-41ac-be3e-fab1204b0320 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:26.318255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffb905d-2a21-41ac-be3e-fab1204b0320 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:26.319914 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477aeff8-4273-418a-87a7-1ff52e04c426 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:26.325282 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bf9455-e329-4918-b4a0-e7a274123c39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:26.325982 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf9455-e329-4918-b4a0-e7a274123c39 service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:26.326267 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf9455-e329-4918-b4a0-e7a274123c39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:26.326536 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf9455-e329-4918-b4a0-e7a274123c39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:26.328689 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-12bf9455-e329-4918-b4a0-e7a274123c39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:26.340958 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bf9455-e329-4918-b4a0-e7a274123c39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:26.342428 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bf9455-e329-4918-b4a0-e7a274123c39 service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:26.342615 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1165/2319] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:21:26 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:21:26.350910 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffb905d-2a21-41ac-be3e-fab1204b0320 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:26.362000 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1155/2320] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:21:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:21:26.380372 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffb905d-2a21-41ac-be3e-fab1204b0320 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:26.395699 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4999f2-e9ca-4921-98ba-a7c3295132f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:26.397443 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4999f2-e9ca-4921-98ba-a7c3295132f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:26.398421 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4999f2-e9ca-4921-98ba-a7c3295132f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:26.400129 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4999f2-e9ca-4921-98ba-a7c3295132f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:26.402155 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4999f2-e9ca-4921-98ba-a7c3295132f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:26.406990 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4999f2-e9ca-4921-98ba-a7c3295132f8 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:26.434437 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1166/2321] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:21:26 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:21:26.438986 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bf9455-e329-4918-b4a0-e7a274123c39 service neutron] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:26.465029 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bf9455-e329-4918-b4a0-e7a274123c39 service neutron] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:26.474201 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b0fed-cfeb-4e38-8292-30ae927cae72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:26.474972 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b0fed-cfeb-4e38-8292-30ae927cae72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:26.475259 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b0fed-cfeb-4e38-8292-30ae927cae72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:26.475700 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b0fed-cfeb-4e38-8292-30ae927cae72 admin admin] PATH_INFO: `/v3/projects/b62ac8f0719d4b899f9d651dc8718e3c/users/50315c1220da4d96919acf627590be91/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:26.515350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6b0fed-cfeb-4e38-8292-30ae927cae72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b62ac8f0719d4b899f9d651dc8718e3c, user_id=50315c1220da4d96919acf627590be91, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:26.519574 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6b0fed-cfeb-4e38-8292-30ae927cae72 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:26.541959 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1156/2322] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:21:26 2022] PUT /identity/v3/projects/b62ac8f0719d4b899f9d651dc8718e3c/users/50315c1220da4d96919acf627590be91/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:21:26.555366 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca9aae-c097-4780-bfa9-4d218c2ff87a None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:26.555833 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca9aae-c097-4780-bfa9-4d218c2ff87a None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:26.556280 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca9aae-c097-4780-bfa9-4d218c2ff87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:26.604395 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-27ca9aae-c097-4780-bfa9-4d218c2ff87a None None] MFA Rules not processed for user `50315c1220da4d96919acf627590be91`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:21:26.610761 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-27ca9aae-c097-4780-bfa9-4d218c2ff87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:26.720801 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1167/2323] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:21:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:21:26.723928 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b0fed-cfeb-4e38-8292-30ae927cae72 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:26.828914 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b0fed-cfeb-4e38-8292-30ae927cae72 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:26.836288 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4400d7d-4d48-4ce1-90a1-876878551c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:26.837165 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4400d7d-4d48-4ce1-90a1-876878551c3b admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:26.837479 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4400d7d-4d48-4ce1-90a1-876878551c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:26.837752 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4400d7d-4d48-4ce1-90a1-876878551c3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:26.838432 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4400d7d-4d48-4ce1-90a1-876878551c3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:26.840479 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4400d7d-4d48-4ce1-90a1-876878551c3b admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:26.874952 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:26.892861 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1157/2324] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:21:26 2022] POST /identity/v3/projects => generated 387 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:21:26.914868 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca9aae-c097-4780-bfa9-4d218c2ff87a None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:26.927859 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca9aae-c097-4780-bfa9-4d218c2ff87a None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:26.986870 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:27.062573 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6280e55f-b85f-44d2-82cb-6a65388fbd57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:27.063530 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280e55f-b85f-44d2-82cb-6a65388fbd57 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:27.063822 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280e55f-b85f-44d2-82cb-6a65388fbd57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:27.064134 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6280e55f-b85f-44d2-82cb-6a65388fbd57 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:27.071065 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6280e55f-b85f-44d2-82cb-6a65388fbd57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:27.071065 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6280e55f-b85f-44d2-82cb-6a65388fbd57 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:27.109429 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1168/2325] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:21:26 2022] POST /identity/v3/users => generated 300 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:21:27.116014 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4400d7d-4d48-4ce1-90a1-876878551c3b admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:27.133717 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4400d7d-4d48-4ce1-90a1-876878551c3b admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:27.138727 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b931970-b5da-4a2d-b129-3a9d1ca649a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:27.139492 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b931970-b5da-4a2d-b129-3a9d1ca649a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:27.139837 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b931970-b5da-4a2d-b129-3a9d1ca649a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:27.140212 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b931970-b5da-4a2d-b129-3a9d1ca649a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:27.141099 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b931970-b5da-4a2d-b129-3a9d1ca649a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:27.142837 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b931970-b5da-4a2d-b129-3a9d1ca649a8 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:27.156962 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1158/2326] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:21:27.160738 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6280e55f-b85f-44d2-82cb-6a65388fbd57 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:27.171779 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6280e55f-b85f-44d2-82cb-6a65388fbd57 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:27.184237 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec64bcc1-b931-40b6-8675-5ddf5b497e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:27.185969 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec64bcc1-b931-40b6-8675-5ddf5b497e54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:27.186354 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec64bcc1-b931-40b6-8675-5ddf5b497e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:27.186870 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec64bcc1-b931-40b6-8675-5ddf5b497e54 admin admin] PATH_INFO: `/v3/projects/42616b9cf7914e1aaf78f6f33688b135/users/6a840823c0b245fabccf8cec6c974076/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:27.204447 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec64bcc1-b931-40b6-8675-5ddf5b497e54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42616b9cf7914e1aaf78f6f33688b135, user_id=6a840823c0b245fabccf8cec6c974076, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:27.209153 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec64bcc1-b931-40b6-8675-5ddf5b497e54 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:27.236481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1169/2327] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:21:27 2022] PUT /identity/v3/projects/42616b9cf7914e1aaf78f6f33688b135/users/6a840823c0b245fabccf8cec6c974076/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:21:27.237724 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b931970-b5da-4a2d-b129-3a9d1ca649a8 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:27.263262 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b931970-b5da-4a2d-b129-3a9d1ca649a8 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:27.406113 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7019f9e-5b54-443b-966e-727af59dc607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:27.407905 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7019f9e-5b54-443b-966e-727af59dc607 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:27.408741 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7019f9e-5b54-443b-966e-727af59dc607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:27.409587 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7019f9e-5b54-443b-966e-727af59dc607 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:27.411150 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7019f9e-5b54-443b-966e-727af59dc607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:27.413912 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7019f9e-5b54-443b-966e-727af59dc607 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:27.437819 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1159/2328] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:21:27.442189 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec64bcc1-b931-40b6-8675-5ddf5b497e54 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:27.458242 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec64bcc1-b931-40b6-8675-5ddf5b497e54 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:27.464992 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee62d8-4c73-43f2-8f07-e6e6c2c61159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:27.467007 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee62d8-4c73-43f2-8f07-e6e6c2c61159 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:27.467007 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee62d8-4c73-43f2-8f07-e6e6c2c61159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:27.467007 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee62d8-4c73-43f2-8f07-e6e6c2c61159 admin admin] PATH_INFO: `/v3/projects/42616b9cf7914e1aaf78f6f33688b135/users/6a840823c0b245fabccf8cec6c974076/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:27.468913 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee62d8-4c73-43f2-8f07-e6e6c2c61159 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42616b9cf7914e1aaf78f6f33688b135, user_id=6a840823c0b245fabccf8cec6c974076, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:27.472087 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee62d8-4c73-43f2-8f07-e6e6c2c61159 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:27.488811 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1170/2329] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:21:27 2022] PUT /identity/v3/projects/42616b9cf7914e1aaf78f6f33688b135/users/6a840823c0b245fabccf8cec6c974076/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:21:27.500985 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2e04b-cd2b-4e69-a24f-ac8d319f8dfc None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:27.501600 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2e04b-cd2b-4e69-a24f-ac8d319f8dfc None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:27.501600 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2e04b-cd2b-4e69-a24f-ac8d319f8dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:27.560512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-96d2e04b-cd2b-4e69-a24f-ac8d319f8dfc None None] MFA Rules not processed for user `6a840823c0b245fabccf8cec6c974076`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:21:27.561808 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-96d2e04b-cd2b-4e69-a24f-ac8d319f8dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:27.687500 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1160/2330] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:21:27 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:21:27.701995 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee62d8-4c73-43f2-8f07-e6e6c2c61159 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:27.705057 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-e4ee62d8-4c73-43f2-8f07-e6e6c2c61159 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:27.810665 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee62d8-4c73-43f2-8f07-e6e6c2c61159 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:27.815477 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98d5242-1ff2-4bbe-9697-b8338065b96d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:27.816353 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d5242-1ff2-4bbe-9697-b8338065b96d service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:27.816879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d5242-1ff2-4bbe-9697-b8338065b96d service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:27.817193 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d5242-1ff2-4bbe-9697-b8338065b96d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:27.829527 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98d5242-1ff2-4bbe-9697-b8338065b96d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:27.831016 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98d5242-1ff2-4bbe-9697-b8338065b96d service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:27.844569 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1171/2331] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:21:27 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:21:28.680431 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2e04b-cd2b-4e69-a24f-ac8d319f8dfc None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:28.912783 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2e04b-cd2b-4e69-a24f-ac8d319f8dfc None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:28.929714 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade02ec-f824-4b67-8150-c4f074710454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:28.931008 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade02ec-f824-4b67-8150-c4f074710454 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:28.931480 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade02ec-f824-4b67-8150-c4f074710454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:28.931912 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade02ec-f824-4b67-8150-c4f074710454 admin admin] PATH_INFO: `/v3/users/50315c1220da4d96919acf627590be91` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:28.934905 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade02ec-f824-4b67-8150-c4f074710454 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50315c1220da4d96919acf627590be91)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:28.940388 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade02ec-f824-4b67-8150-c4f074710454 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:28.988321 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:29.097245 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-5ade02ec-f824-4b67-8150-c4f074710454 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50315c1220da4d96919acf627590be91'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:29.113807 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-5ade02ec-f824-4b67-8150-c4f074710454 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50315c1220da4d96919acf627590be91'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:29.114568 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-5ade02ec-f824-4b67-8150-c4f074710454 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50315c1220da4d96919acf627590be91'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:29.115187 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-5ade02ec-f824-4b67-8150-c4f074710454 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50315c1220da4d96919acf627590be91'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:29.136908 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-5ade02ec-f824-4b67-8150-c4f074710454 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50315c1220da4d96919acf627590be91'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:29.171467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1161/2332] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:21:28 2022] DELETE /identity/v3/users/50315c1220da4d96919acf627590be91 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:21:29.176252 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c920e65-abf4-4d74-a8d7-e829272d1839 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:29.176779 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c920e65-abf4-4d74-a8d7-e829272d1839 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:29.176884 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c920e65-abf4-4d74-a8d7-e829272d1839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:29.206477 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-9c920e65-abf4-4d74-a8d7-e829272d1839 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:21:29.213899 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-9c920e65-abf4-4d74-a8d7-e829272d1839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:29.325090 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1172/2333] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:21:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:21:29.343277 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade02ec-f824-4b67-8150-c4f074710454 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:29.346950 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-5ade02ec-f824-4b67-8150-c4f074710454 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:29.464280 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade02ec-f824-4b67-8150-c4f074710454 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:29.470724 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86ad72c-500c-453a-89f3-8b6cabacb5f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:29.471459 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ad72c-500c-453a-89f3-8b6cabacb5f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:29.471748 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ad72c-500c-453a-89f3-8b6cabacb5f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:29.472024 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86ad72c-500c-453a-89f3-8b6cabacb5f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:29.483972 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86ad72c-500c-453a-89f3-8b6cabacb5f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:29.485610 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86ad72c-500c-453a-89f3-8b6cabacb5f9 service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:29.516612 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1162/2334] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:21:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:21:30.428224 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c920e65-abf4-4d74-a8d7-e829272d1839 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:30.429859 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-9c920e65-abf4-4d74-a8d7-e829272d1839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:30.446565 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c920e65-abf4-4d74-a8d7-e829272d1839 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:30.453033 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9aee81-5467-4fe7-b28a-a8f04435014d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:30.454547 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9aee81-5467-4fe7-b28a-a8f04435014d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:30.454547 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9aee81-5467-4fe7-b28a-a8f04435014d admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:30.454784 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9aee81-5467-4fe7-b28a-a8f04435014d admin admin] PATH_INFO: `/v3/projects/b62ac8f0719d4b899f9d651dc8718e3c` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:30.467337 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9aee81-5467-4fe7-b28a-a8f04435014d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b62ac8f0719d4b899f9d651dc8718e3c)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:30.473012 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9aee81-5467-4fe7-b28a-a8f04435014d admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:30.516078 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:30.576732 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-2a9aee81-5467-4fe7-b28a-a8f04435014d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b62ac8f0719d4b899f9d651dc8718e3c', 'request_id': 'req-2a9aee81-5467-4fe7-b28a-a8f04435014d'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:30.585593 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-2a9aee81-5467-4fe7-b28a-a8f04435014d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b62ac8f0719d4b899f9d651dc8718e3c', 'request_id': 'req-2a9aee81-5467-4fe7-b28a-a8f04435014d'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:30.596784 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-2a9aee81-5467-4fe7-b28a-a8f04435014d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b62ac8f0719d4b899f9d651dc8718e3c', 'request_id': 'req-2a9aee81-5467-4fe7-b28a-a8f04435014d'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:30.612031 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-2a9aee81-5467-4fe7-b28a-a8f04435014d admin admin] The token cache is being invalidate because project b62ac8f0719d4b899f9d651dc8718e3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:21:30.612031 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-2a9aee81-5467-4fe7-b28a-a8f04435014d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:30.612387 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-2a9aee81-5467-4fe7-b28a-a8f04435014d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:30.614410 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1173/2335] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:21:30 2022] DELETE /identity/v3/projects/b62ac8f0719d4b899f9d651dc8718e3c => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:21:30.619029 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86ad72c-500c-453a-89f3-8b6cabacb5f9 service neutron] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:30.630793 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86ad72c-500c-453a-89f3-8b6cabacb5f9 service neutron] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:30.727207 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e7745-cef6-481b-ba6b-7305a8d4f231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:30.728777 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e7745-cef6-481b-ba6b-7305a8d4f231 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:30.729160 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e7745-cef6-481b-ba6b-7305a8d4f231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:30.729440 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e7745-cef6-481b-ba6b-7305a8d4f231 admin admin] PATH_INFO: `/v3/users/6a840823c0b245fabccf8cec6c974076` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:30.731573 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e7745-cef6-481b-ba6b-7305a8d4f231 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a840823c0b245fabccf8cec6c974076)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:30.733808 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e7745-cef6-481b-ba6b-7305a8d4f231 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:30.818162 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:30.832828 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c19e7745-cef6-481b-ba6b-7305a8d4f231 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a840823c0b245fabccf8cec6c974076'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:30.849171 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c19e7745-cef6-481b-ba6b-7305a8d4f231 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a840823c0b245fabccf8cec6c974076'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:30.850183 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c19e7745-cef6-481b-ba6b-7305a8d4f231 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a840823c0b245fabccf8cec6c974076'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:30.852089 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c19e7745-cef6-481b-ba6b-7305a8d4f231 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a840823c0b245fabccf8cec6c974076'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:30.867390 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c19e7745-cef6-481b-ba6b-7305a8d4f231 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a840823c0b245fabccf8cec6c974076'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:30.908875 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1163/2336] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:21:30 2022] DELETE /identity/v3/users/6a840823c0b245fabccf8cec6c974076 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:21:31.449010 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9aee81-5467-4fe7-b28a-a8f04435014d admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:31.451513 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-2a9aee81-5467-4fe7-b28a-a8f04435014d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:31.478914 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9aee81-5467-4fe7-b28a-a8f04435014d admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:31.590424 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1161fd89-701c-469f-b13c-aff7a95bfc8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:31.591731 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1161fd89-701c-469f-b13c-aff7a95bfc8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:31.591731 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1161fd89-701c-469f-b13c-aff7a95bfc8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:31.592023 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1161fd89-701c-469f-b13c-aff7a95bfc8c admin admin] PATH_INFO: `/v3/projects/42616b9cf7914e1aaf78f6f33688b135` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:31.593238 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1161fd89-701c-469f-b13c-aff7a95bfc8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=42616b9cf7914e1aaf78f6f33688b135)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:31.595245 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1161fd89-701c-469f-b13c-aff7a95bfc8c admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:31.710062 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-1161fd89-701c-469f-b13c-aff7a95bfc8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42616b9cf7914e1aaf78f6f33688b135', 'request_id': 'req-1161fd89-701c-469f-b13c-aff7a95bfc8c'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:31.719342 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-1161fd89-701c-469f-b13c-aff7a95bfc8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42616b9cf7914e1aaf78f6f33688b135', 'request_id': 'req-1161fd89-701c-469f-b13c-aff7a95bfc8c'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:31.729638 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-1161fd89-701c-469f-b13c-aff7a95bfc8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42616b9cf7914e1aaf78f6f33688b135', 'request_id': 'req-1161fd89-701c-469f-b13c-aff7a95bfc8c'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:31.741960 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-1161fd89-701c-469f-b13c-aff7a95bfc8c admin admin] The token cache is being invalidate because project 42616b9cf7914e1aaf78f6f33688b135 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:21:31.742440 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-1161fd89-701c-469f-b13c-aff7a95bfc8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:31.743513 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-1161fd89-701c-469f-b13c-aff7a95bfc8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:31.752506 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1174/2337] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:21:31 2022] DELETE /identity/v3/projects/42616b9cf7914e1aaf78f6f33688b135 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:21:31.755475 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e16cc5-861e-4211-bf87-73d8f8cf284a None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:31.756441 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e16cc5-861e-4211-bf87-73d8f8cf284a None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:31.757204 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e16cc5-861e-4211-bf87-73d8f8cf284a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:31.790114 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-23e16cc5-861e-4211-bf87-73d8f8cf284a None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:21:31.791388 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-23e16cc5-861e-4211-bf87-73d8f8cf284a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:31.911623 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1164/2338] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:21:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:21:31.921160 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87337f2d-8d80-400e-8c90-30f3a184275e None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:31.921482 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87337f2d-8d80-400e-8c90-30f3a184275e None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:31.921961 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87337f2d-8d80-400e-8c90-30f3a184275e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:31.946982 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-87337f2d-8d80-400e-8c90-30f3a184275e None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:21:31.948569 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-87337f2d-8d80-400e-8c90-30f3a184275e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:31.964983 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1175/2339] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:21:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:21:31.968965 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e16cc5-861e-4211-bf87-73d8f8cf284a None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:31.987219 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e16cc5-861e-4211-bf87-73d8f8cf284a None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:31.993055 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4222e0-a319-4b85-b2b6-f2265dc7740f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:31.994586 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4222e0-a319-4b85-b2b6-f2265dc7740f admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:31.994586 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4222e0-a319-4b85-b2b6-f2265dc7740f admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:31.994586 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4222e0-a319-4b85-b2b6-f2265dc7740f admin admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:31.995438 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4222e0-a319-4b85-b2b6-f2265dc7740f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:21:31.995794 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4222e0-a319-4b85-b2b6-f2265dc7740f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:31.998014 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4222e0-a319-4b85-b2b6-f2265dc7740f admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:32.034485 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1165/2340] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:21:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:21:32.036500 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87337f2d-8d80-400e-8c90-30f3a184275e None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:32.055680 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87337f2d-8d80-400e-8c90-30f3a184275e None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:32.063903 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccb049a-98be-4d98-bfc0-cb62955447e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:32.065119 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb049a-98be-4d98-bfc0-cb62955447e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:32.065546 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb049a-98be-4d98-bfc0-cb62955447e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:32.065920 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb049a-98be-4d98-bfc0-cb62955447e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:32.066979 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccb049a-98be-4d98-bfc0-cb62955447e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:32.070968 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccb049a-98be-4d98-bfc0-cb62955447e0 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:32.120689 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1176/2341] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:21:32 2022] POST /identity/v3/projects => generated 381 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:21:32.132396 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4222e0-a319-4b85-b2b6-f2265dc7740f admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:32.144440 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4222e0-a319-4b85-b2b6-f2265dc7740f admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:32.251598 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1dc838-3a9b-4941-9ec4-27d77940bc98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:32.252629 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1dc838-3a9b-4941-9ec4-27d77940bc98 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:32.253133 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1dc838-3a9b-4941-9ec4-27d77940bc98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:32.253631 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1dc838-3a9b-4941-9ec4-27d77940bc98 admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:32.255017 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1dc838-3a9b-4941-9ec4-27d77940bc98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:32.258371 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1dc838-3a9b-4941-9ec4-27d77940bc98 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:32.297221 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1166/2342] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:21:32 2022] POST /identity/v3/users => generated 291 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:21:32.302146 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccb049a-98be-4d98-bfc0-cb62955447e0 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:32.313626 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:32.326686 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccb049a-98be-4d98-bfc0-cb62955447e0 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:32.338800 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d2a44a-0764-4187-8031-1f2ffb8234ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:32.339497 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2a44a-0764-4187-8031-1f2ffb8234ea admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:32.339771 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2a44a-0764-4187-8031-1f2ffb8234ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:32.340038 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2a44a-0764-4187-8031-1f2ffb8234ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:32.340785 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d2a44a-0764-4187-8031-1f2ffb8234ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:32.342432 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d2a44a-0764-4187-8031-1f2ffb8234ea admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:32.354808 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1177/2343] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:21:32 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:21:32.359549 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1dc838-3a9b-4941-9ec4-27d77940bc98 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:32.379124 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1dc838-3a9b-4941-9ec4-27d77940bc98 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:32.387152 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797209e4-17b8-4934-8b0b-11c5e0bc4eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:32.388325 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797209e4-17b8-4934-8b0b-11c5e0bc4eff admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:32.388820 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797209e4-17b8-4934-8b0b-11c5e0bc4eff admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:32.389844 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797209e4-17b8-4934-8b0b-11c5e0bc4eff admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:32.390986 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797209e4-17b8-4934-8b0b-11c5e0bc4eff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:32.394625 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797209e4-17b8-4934-8b0b-11c5e0bc4eff admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:32.411683 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1167/2344] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:21:32 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:21:32.416638 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d2a44a-0764-4187-8031-1f2ffb8234ea admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:32.441314 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d2a44a-0764-4187-8031-1f2ffb8234ea admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:32.446387 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e578b3d-812c-4aea-85be-4ac4d8372f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:32.447437 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e578b3d-812c-4aea-85be-4ac4d8372f6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:32.447718 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e578b3d-812c-4aea-85be-4ac4d8372f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:32.447996 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e578b3d-812c-4aea-85be-4ac4d8372f6e admin admin] PATH_INFO: `/v3/projects/9c1b3bba275245b692f35e07f3ce62dd/users/d6e02c78eece4ef3a454b195a233e67f/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:32.467505 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e578b3d-812c-4aea-85be-4ac4d8372f6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c1b3bba275245b692f35e07f3ce62dd, user_id=d6e02c78eece4ef3a454b195a233e67f, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:32.470255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e578b3d-812c-4aea-85be-4ac4d8372f6e admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:32.487430 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1178/2345] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:21:32 2022] PUT /identity/v3/projects/9c1b3bba275245b692f35e07f3ce62dd/users/d6e02c78eece4ef3a454b195a233e67f/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:21:32.499773 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab548d2b-235e-4a65-ac67-442468a239a9 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:32.500200 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab548d2b-235e-4a65-ac67-442468a239a9 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:32.500570 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab548d2b-235e-4a65-ac67-442468a239a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:32.537424 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-ab548d2b-235e-4a65-ac67-442468a239a9 None None] MFA Rules not processed for user `d6e02c78eece4ef3a454b195a233e67f`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:21:32.538921 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-ab548d2b-235e-4a65-ac67-442468a239a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:32.548872 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:32.638883 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1168/2346] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:21:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:21:32.644616 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e578b3d-812c-4aea-85be-4ac4d8372f6e admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:32.732679 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e578b3d-812c-4aea-85be-4ac4d8372f6e admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:32.738155 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a7b0e7-1389-4c28-b80c-0bc477fe74f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:32.738956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7b0e7-1389-4c28-b80c-0bc477fe74f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:32.739304 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7b0e7-1389-4c28-b80c-0bc477fe74f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:32.739651 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7b0e7-1389-4c28-b80c-0bc477fe74f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:32.740422 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a7b0e7-1389-4c28-b80c-0bc477fe74f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:32.742512 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a7b0e7-1389-4c28-b80c-0bc477fe74f3 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:32.785067 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1179/2347] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:21:32 2022] POST /identity/v3/projects => generated 381 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:21:32.788377 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab548d2b-235e-4a65-ac67-442468a239a9 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:32.801958 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab548d2b-235e-4a65-ac67-442468a239a9 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:32.872011 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aae39d9-1a68-444b-b329-b4fb52b6e589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:32.872692 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aae39d9-1a68-444b-b329-b4fb52b6e589 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:32.873019 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aae39d9-1a68-444b-b329-b4fb52b6e589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:32.873269 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aae39d9-1a68-444b-b329-b4fb52b6e589 admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:32.873962 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aae39d9-1a68-444b-b329-b4fb52b6e589 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:32.875868 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aae39d9-1a68-444b-b329-b4fb52b6e589 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:32.897653 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1169/2348] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:21:32 2022] POST /identity/v3/users => generated 297 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:21:32.902874 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a7b0e7-1389-4c28-b80c-0bc477fe74f3 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:32.913614 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a7b0e7-1389-4c28-b80c-0bc477fe74f3 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:32.918667 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3965d2b7-4247-4708-a13c-7f2e27255369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:32.919337 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965d2b7-4247-4708-a13c-7f2e27255369 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:32.919619 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965d2b7-4247-4708-a13c-7f2e27255369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:32.919895 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965d2b7-4247-4708-a13c-7f2e27255369 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:32.920668 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3965d2b7-4247-4708-a13c-7f2e27255369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:32.922330 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3965d2b7-4247-4708-a13c-7f2e27255369 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:32.933660 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1180/2349] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:21:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:21:32.938215 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aae39d9-1a68-444b-b329-b4fb52b6e589 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:32.949054 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aae39d9-1a68-444b-b329-b4fb52b6e589 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:32.953926 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f2652-0ecc-4a4a-80dc-e1c3848ece2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:32.954874 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f2652-0ecc-4a4a-80dc-e1c3848ece2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:32.955155 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f2652-0ecc-4a4a-80dc-e1c3848ece2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:32.955456 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029f2652-0ecc-4a4a-80dc-e1c3848ece2c admin admin] PATH_INFO: `/v3/projects/10060087b0b24f169f900afff8cbedb3/users/b9c9799e48a14c63b8f303145a7da8fb/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:32.972971 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029f2652-0ecc-4a4a-80dc-e1c3848ece2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=10060087b0b24f169f900afff8cbedb3, user_id=b9c9799e48a14c63b8f303145a7da8fb, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:32.975693 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029f2652-0ecc-4a4a-80dc-e1c3848ece2c admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:32.991798 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1170/2350] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:21:32 2022] PUT /identity/v3/projects/10060087b0b24f169f900afff8cbedb3/users/b9c9799e48a14c63b8f303145a7da8fb/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:21:32.996377 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3965d2b7-4247-4708-a13c-7f2e27255369 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:33.007713 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3965d2b7-4247-4708-a13c-7f2e27255369 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:33.070201 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a88d6-5515-4511-8b20-035ab75becab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:33.070957 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a88d6-5515-4511-8b20-035ab75becab admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:33.071235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a88d6-5515-4511-8b20-035ab75becab admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:33.071504 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a88d6-5515-4511-8b20-035ab75becab admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:33.072238 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24a88d6-5515-4511-8b20-035ab75becab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:33.073916 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24a88d6-5515-4511-8b20-035ab75becab admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:33.085860 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1181/2351] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:21:32 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:21:33.090359 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f2652-0ecc-4a4a-80dc-e1c3848ece2c admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:33.110226 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029f2652-0ecc-4a4a-80dc-e1c3848ece2c admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:33.118844 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69604671-6435-41e2-addc-c73f72b68a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:33.120050 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69604671-6435-41e2-addc-c73f72b68a82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:33.120338 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69604671-6435-41e2-addc-c73f72b68a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:33.120700 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69604671-6435-41e2-addc-c73f72b68a82 admin admin] PATH_INFO: `/v3/projects/10060087b0b24f169f900afff8cbedb3/users/b9c9799e48a14c63b8f303145a7da8fb/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:33.122839 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69604671-6435-41e2-addc-c73f72b68a82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10060087b0b24f169f900afff8cbedb3, user_id=b9c9799e48a14c63b8f303145a7da8fb, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:33.125556 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69604671-6435-41e2-addc-c73f72b68a82 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:33.136977 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1171/2352] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:21:33 2022] PUT /identity/v3/projects/10060087b0b24f169f900afff8cbedb3/users/b9c9799e48a14c63b8f303145a7da8fb/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:21:33.147364 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b3a48-e813-4d58-a242-d04df579e654 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:33.147666 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b3a48-e813-4d58-a242-d04df579e654 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:33.147944 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b3a48-e813-4d58-a242-d04df579e654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:33.176696 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-8a9b3a48-e813-4d58-a242-d04df579e654 None None] MFA Rules not processed for user `b9c9799e48a14c63b8f303145a7da8fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:21:33.177926 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-8a9b3a48-e813-4d58-a242-d04df579e654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:33.222784 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:33.265268 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1182/2353] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:21:33 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:21:33.277658 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69604671-6435-41e2-addc-c73f72b68a82 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:33.279488 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-69604671-6435-41e2-addc-c73f72b68a82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:33.346159 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69604671-6435-41e2-addc-c73f72b68a82 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:33.351115 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a434962-b2d0-4954-89ff-3d1c0485772c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:33.351479 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a434962-b2d0-4954-89ff-3d1c0485772c service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:33.351774 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a434962-b2d0-4954-89ff-3d1c0485772c service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:33.352117 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a434962-b2d0-4954-89ff-3d1c0485772c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:33.363799 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a434962-b2d0-4954-89ff-3d1c0485772c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:33.365453 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a434962-b2d0-4954-89ff-3d1c0485772c service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:33.378484 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1172/2354] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:21:33 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:21:33.478118 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b3a48-e813-4d58-a242-d04df579e654 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:33.489413 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b3a48-e813-4d58-a242-d04df579e654 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:33.493448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436bbb9-9264-4c89-ad43-4d967ab33384 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:33.494116 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436bbb9-9264-4c89-ad43-4d967ab33384 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:33.494400 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436bbb9-9264-4c89-ad43-4d967ab33384 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:33.494676 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436bbb9-9264-4c89-ad43-4d967ab33384 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:33.565758 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9436bbb9-9264-4c89-ad43-4d967ab33384 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:33.567272 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9436bbb9-9264-4c89-ad43-4d967ab33384 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:33.578896 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1183/2355] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:21:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:21:33.893394 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a434962-b2d0-4954-89ff-3d1c0485772c service cinder] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:33.902292 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:33.974471 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a434962-b2d0-4954-89ff-3d1c0485772c service cinder] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:33.980080 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d5a509-4abb-428b-b6ca-4c76bdd32911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:33.981165 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5a509-4abb-428b-b6ca-4c76bdd32911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:33.981610 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5a509-4abb-428b-b6ca-4c76bdd32911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:33.981882 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5a509-4abb-428b-b6ca-4c76bdd32911 admin admin] PATH_INFO: `/v3/users/d6e02c78eece4ef3a454b195a233e67f` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:33.983148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d5a509-4abb-428b-b6ca-4c76bdd32911 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6e02c78eece4ef3a454b195a233e67f)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:33.985172 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d5a509-4abb-428b-b6ca-4c76bdd32911 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:34.052130 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-d0d5a509-4abb-428b-b6ca-4c76bdd32911 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6e02c78eece4ef3a454b195a233e67f'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:34.063461 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-d0d5a509-4abb-428b-b6ca-4c76bdd32911 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6e02c78eece4ef3a454b195a233e67f'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:34.064246 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-d0d5a509-4abb-428b-b6ca-4c76bdd32911 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6e02c78eece4ef3a454b195a233e67f'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:34.064883 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-d0d5a509-4abb-428b-b6ca-4c76bdd32911 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6e02c78eece4ef3a454b195a233e67f'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:34.079150 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-d0d5a509-4abb-428b-b6ca-4c76bdd32911 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6e02c78eece4ef3a454b195a233e67f'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:34.097967 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1173/2356] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:21:33 2022] DELETE /identity/v3/users/d6e02c78eece4ef3a454b195a233e67f => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:21:34.102835 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d91af1-9b3d-491a-8b0e-d53828768d08 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:34.103133 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d91af1-9b3d-491a-8b0e-d53828768d08 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:34.103413 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d91af1-9b3d-491a-8b0e-d53828768d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:34.120998 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-d3d91af1-9b3d-491a-8b0e-d53828768d08 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:21:34.122146 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d3d91af1-9b3d-491a-8b0e-d53828768d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:34.185732 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1184/2357] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:21:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:21:34.195905 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d5a509-4abb-428b-b6ca-4c76bdd32911 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:34.197882 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-d0d5a509-4abb-428b-b6ca-4c76bdd32911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:34.258951 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d5a509-4abb-428b-b6ca-4c76bdd32911 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:34.263796 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac93076-233b-4bd2-af11-b8cd567a2a37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:34.264479 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac93076-233b-4bd2-af11-b8cd567a2a37 service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:34.264776 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac93076-233b-4bd2-af11-b8cd567a2a37 service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:34.265076 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac93076-233b-4bd2-af11-b8cd567a2a37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:34.274998 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac93076-233b-4bd2-af11-b8cd567a2a37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:34.276452 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac93076-233b-4bd2-af11-b8cd567a2a37 service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:34.286463 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1174/2358] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:21:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:21:34.571474 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d91af1-9b3d-491a-8b0e-d53828768d08 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:34.572660 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d3d91af1-9b3d-491a-8b0e-d53828768d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:34.586293 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d91af1-9b3d-491a-8b0e-d53828768d08 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:34.592973 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a5997-fc1d-4bc6-b915-296f8de34a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:34.593905 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a5997-fc1d-4bc6-b915-296f8de34a71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:34.594162 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a5997-fc1d-4bc6-b915-296f8de34a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:34.594460 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a5997-fc1d-4bc6-b915-296f8de34a71 admin admin] PATH_INFO: `/v3/projects/9c1b3bba275245b692f35e07f3ce62dd` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:34.595777 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38a5997-fc1d-4bc6-b915-296f8de34a71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c1b3bba275245b692f35e07f3ce62dd)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:34.598028 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38a5997-fc1d-4bc6-b915-296f8de34a71 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:34.651650 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c38a5997-fc1d-4bc6-b915-296f8de34a71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c1b3bba275245b692f35e07f3ce62dd', 'request_id': 'req-c38a5997-fc1d-4bc6-b915-296f8de34a71'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:34.657930 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c38a5997-fc1d-4bc6-b915-296f8de34a71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c1b3bba275245b692f35e07f3ce62dd', 'request_id': 'req-c38a5997-fc1d-4bc6-b915-296f8de34a71'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:34.661641 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:34.666558 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c38a5997-fc1d-4bc6-b915-296f8de34a71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c1b3bba275245b692f35e07f3ce62dd', 'request_id': 'req-c38a5997-fc1d-4bc6-b915-296f8de34a71'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:34.678243 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c38a5997-fc1d-4bc6-b915-296f8de34a71 admin admin] The token cache is being invalidate because project 9c1b3bba275245b692f35e07f3ce62dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:21:34.678531 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c38a5997-fc1d-4bc6-b915-296f8de34a71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:34.679152 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c38a5997-fc1d-4bc6-b915-296f8de34a71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:34.681476 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1185/2359] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:21:34 2022] DELETE /identity/v3/projects/9c1b3bba275245b692f35e07f3ce62dd => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:21:34.685493 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac93076-233b-4bd2-af11-b8cd567a2a37 service neutron] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:34.695893 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac93076-233b-4bd2-af11-b8cd567a2a37 service neutron] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:34.718974 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:34.759349 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ffccaa-3668-4aeb-9416-43034ef9f9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:34.760401 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ffccaa-3668-4aeb-9416-43034ef9f9cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:34.760668 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ffccaa-3668-4aeb-9416-43034ef9f9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:34.761097 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ffccaa-3668-4aeb-9416-43034ef9f9cd admin admin] PATH_INFO: `/v3/users/b9c9799e48a14c63b8f303145a7da8fb` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:34.762330 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ffccaa-3668-4aeb-9416-43034ef9f9cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9c9799e48a14c63b8f303145a7da8fb)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:34.764311 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ffccaa-3668-4aeb-9416-43034ef9f9cd admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:34.811692 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-d6ffccaa-3668-4aeb-9416-43034ef9f9cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9c9799e48a14c63b8f303145a7da8fb'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:34.818667 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-d6ffccaa-3668-4aeb-9416-43034ef9f9cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9c9799e48a14c63b8f303145a7da8fb'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:34.819342 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-d6ffccaa-3668-4aeb-9416-43034ef9f9cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9c9799e48a14c63b8f303145a7da8fb'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:34.819980 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-d6ffccaa-3668-4aeb-9416-43034ef9f9cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9c9799e48a14c63b8f303145a7da8fb'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:34.828141 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-d6ffccaa-3668-4aeb-9416-43034ef9f9cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9c9799e48a14c63b8f303145a7da8fb'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:34.848657 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1175/2360] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:21:34 2022] DELETE /identity/v3/users/b9c9799e48a14c63b8f303145a7da8fb => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:21:35.146142 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a5997-fc1d-4bc6-b915-296f8de34a71 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:35.147598 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-c38a5997-fc1d-4bc6-b915-296f8de34a71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:35.161309 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38a5997-fc1d-4bc6-b915-296f8de34a71 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:35.232664 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5242796d-204c-4fb7-959c-e0969fd187ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:35.233571 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242796d-204c-4fb7-959c-e0969fd187ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:35.233837 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242796d-204c-4fb7-959c-e0969fd187ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:35.234264 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242796d-204c-4fb7-959c-e0969fd187ab admin admin] PATH_INFO: `/v3/projects/10060087b0b24f169f900afff8cbedb3` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:35.235311 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5242796d-204c-4fb7-959c-e0969fd187ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=10060087b0b24f169f900afff8cbedb3)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:35.237273 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5242796d-204c-4fb7-959c-e0969fd187ab admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:35.288565 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-5242796d-204c-4fb7-959c-e0969fd187ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10060087b0b24f169f900afff8cbedb3', 'request_id': 'req-5242796d-204c-4fb7-959c-e0969fd187ab'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:35.295034 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-5242796d-204c-4fb7-959c-e0969fd187ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10060087b0b24f169f900afff8cbedb3', 'request_id': 'req-5242796d-204c-4fb7-959c-e0969fd187ab'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:35.303443 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-5242796d-204c-4fb7-959c-e0969fd187ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10060087b0b24f169f900afff8cbedb3', 'request_id': 'req-5242796d-204c-4fb7-959c-e0969fd187ab'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:35.314674 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-5242796d-204c-4fb7-959c-e0969fd187ab admin admin] The token cache is being invalidate because project 10060087b0b24f169f900afff8cbedb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:21:35.314961 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-5242796d-204c-4fb7-959c-e0969fd187ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:35.315604 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-5242796d-204c-4fb7-959c-e0969fd187ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:35.317241 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1186/2361] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:21:35 2022] DELETE /identity/v3/projects/10060087b0b24f169f900afff8cbedb3 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:21:35.322432 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f44b8d-bb8d-40a1-823b-0bb56b19e287 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:35.322939 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f44b8d-bb8d-40a1-823b-0bb56b19e287 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:35.323036 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f44b8d-bb8d-40a1-823b-0bb56b19e287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:35.341713 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-a0f44b8d-bb8d-40a1-823b-0bb56b19e287 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:21:35.342887 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-a0f44b8d-bb8d-40a1-823b-0bb56b19e287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:35.405411 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1176/2362] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:21:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:21:35.415502 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6866c-b266-4cb7-a1e5-13771fe6c83b None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:35.415800 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6866c-b266-4cb7-a1e5-13771fe6c83b None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:35.416065 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6866c-b266-4cb7-a1e5-13771fe6c83b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:35.435745 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-08a6866c-b266-4cb7-a1e5-13771fe6c83b None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:21:35.437033 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-08a6866c-b266-4cb7-a1e5-13771fe6c83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:35.445425 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1187/2363] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:21:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:21:35.450025 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f44b8d-bb8d-40a1-823b-0bb56b19e287 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:35.460674 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f44b8d-bb8d-40a1-823b-0bb56b19e287 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:35.466196 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade45ce-e1b1-4c0d-856b-bfc3ae47fa91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:35.466880 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade45ce-e1b1-4c0d-856b-bfc3ae47fa91 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:35.467161 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade45ce-e1b1-4c0d-856b-bfc3ae47fa91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:35.467440 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade45ce-e1b1-4c0d-856b-bfc3ae47fa91 admin admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:35.468174 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ade45ce-e1b1-4c0d-856b-bfc3ae47fa91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:21:35.468479 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ade45ce-e1b1-4c0d-856b-bfc3ae47fa91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:35.470115 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ade45ce-e1b1-4c0d-856b-bfc3ae47fa91 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:35.483811 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1177/2364] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:21:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:21:35.488348 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a6866c-b266-4cb7-a1e5-13771fe6c83b None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:35.499980 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a6866c-b266-4cb7-a1e5-13771fe6c83b None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:35.505421 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a35f878-4f69-4076-b93d-b11ece109c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:35.506060 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a35f878-4f69-4076-b93d-b11ece109c30 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:35.506325 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a35f878-4f69-4076-b93d-b11ece109c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:35.506614 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a35f878-4f69-4076-b93d-b11ece109c30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:35.507302 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a35f878-4f69-4076-b93d-b11ece109c30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:35.510001 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a35f878-4f69-4076-b93d-b11ece109c30 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:35.542003 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1188/2365] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:21:35 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:21:35.546723 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade45ce-e1b1-4c0d-856b-bfc3ae47fa91 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:35.557401 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade45ce-e1b1-4c0d-856b-bfc3ae47fa91 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:35.589096 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:35.620547 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d7cf20-0cf2-448e-a9db-bc067cefd455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:35.621297 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7cf20-0cf2-448e-a9db-bc067cefd455 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:35.621577 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7cf20-0cf2-448e-a9db-bc067cefd455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:35.621855 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7cf20-0cf2-448e-a9db-bc067cefd455 admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:35.622635 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d7cf20-0cf2-448e-a9db-bc067cefd455 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:35.624652 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d7cf20-0cf2-448e-a9db-bc067cefd455 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:35.647836 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1178/2366] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:21:35 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:21:35.649592 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a35f878-4f69-4076-b93d-b11ece109c30 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:35.660627 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a35f878-4f69-4076-b93d-b11ece109c30 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:35.666411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091fa5f5-59b9-42e2-9222-0db62cf1245e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:35.667089 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fa5f5-59b9-42e2-9222-0db62cf1245e admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:35.667370 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fa5f5-59b9-42e2-9222-0db62cf1245e admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:35.667628 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fa5f5-59b9-42e2-9222-0db62cf1245e admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:35.668396 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091fa5f5-59b9-42e2-9222-0db62cf1245e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:35.670060 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091fa5f5-59b9-42e2-9222-0db62cf1245e admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:35.684708 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1189/2367] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:21:35.686215 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d7cf20-0cf2-448e-a9db-bc067cefd455 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:35.697659 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d7cf20-0cf2-448e-a9db-bc067cefd455 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:35.703452 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3fe2ca-bbec-4ad8-b8eb-e572321ebeea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:35.704250 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3fe2ca-bbec-4ad8-b8eb-e572321ebeea admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:35.704412 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3fe2ca-bbec-4ad8-b8eb-e572321ebeea admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:35.704688 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3fe2ca-bbec-4ad8-b8eb-e572321ebeea admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:35.705450 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3fe2ca-bbec-4ad8-b8eb-e572321ebeea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:35.707171 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3fe2ca-bbec-4ad8-b8eb-e572321ebeea admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:35.719726 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1179/2368] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:21:35.723670 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091fa5f5-59b9-42e2-9222-0db62cf1245e admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:35.735405 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091fa5f5-59b9-42e2-9222-0db62cf1245e admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:35.740372 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242adea9-fe1b-4d41-bdb1-c16c354ac84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:35.741352 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242adea9-fe1b-4d41-bdb1-c16c354ac84e admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:35.741635 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242adea9-fe1b-4d41-bdb1-c16c354ac84e admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:35.741912 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242adea9-fe1b-4d41-bdb1-c16c354ac84e admin admin] PATH_INFO: `/v3/projects/ff00c4faa4754b2b8dcb1bad19806864/users/a693560de0ab492eaf0766fe84b01acd/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:35.761229 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242adea9-fe1b-4d41-bdb1-c16c354ac84e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff00c4faa4754b2b8dcb1bad19806864, user_id=a693560de0ab492eaf0766fe84b01acd, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:35.764014 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242adea9-fe1b-4d41-bdb1-c16c354ac84e admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:35.778854 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1190/2369] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:21:35 2022] PUT /identity/v3/projects/ff00c4faa4754b2b8dcb1bad19806864/users/a693560de0ab492eaf0766fe84b01acd/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:21:35.789460 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eebfed-7b3a-427a-8138-aa5a7dac72d3 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:35.789733 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eebfed-7b3a-427a-8138-aa5a7dac72d3 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:35.790060 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eebfed-7b3a-427a-8138-aa5a7dac72d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:35.818254 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-81eebfed-7b3a-427a-8138-aa5a7dac72d3 None None] MFA Rules not processed for user `a693560de0ab492eaf0766fe84b01acd`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:21:35.819513 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-81eebfed-7b3a-427a-8138-aa5a7dac72d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:35.886586 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1180/2370] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:21:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:21:35.891279 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242adea9-fe1b-4d41-bdb1-c16c354ac84e admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:35.905784 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:35.960676 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242adea9-fe1b-4d41-bdb1-c16c354ac84e admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:35.966113 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d58652-4937-42c0-83e5-af8259a4bf32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:35.966837 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d58652-4937-42c0-83e5-af8259a4bf32 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:35.967118 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d58652-4937-42c0-83e5-af8259a4bf32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:35.967394 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d58652-4937-42c0-83e5-af8259a4bf32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:35.968107 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d58652-4937-42c0-83e5-af8259a4bf32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:35.970181 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d58652-4937-42c0-83e5-af8259a4bf32 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:36.001124 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1191/2371] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:21:35 2022] POST /identity/v3/projects => generated 375 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:21:36.005699 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eebfed-7b3a-427a-8138-aa5a7dac72d3 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:36.017628 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eebfed-7b3a-427a-8138-aa5a7dac72d3 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:36.091805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab493122-b6c1-4dcd-9c17-ac3312987900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:36.092501 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab493122-b6c1-4dcd-9c17-ac3312987900 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:36.092783 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab493122-b6c1-4dcd-9c17-ac3312987900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:36.093105 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab493122-b6c1-4dcd-9c17-ac3312987900 admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:36.094108 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab493122-b6c1-4dcd-9c17-ac3312987900 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:36.095766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab493122-b6c1-4dcd-9c17-ac3312987900 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:36.121657 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1181/2372] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:21:36 2022] POST /identity/v3/users => generated 294 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:21:36.126644 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d58652-4937-42c0-83e5-af8259a4bf32 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:36.138244 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d58652-4937-42c0-83e5-af8259a4bf32 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:36.143655 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927586b-71f3-4692-b18b-6622e225c091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:36.145049 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927586b-71f3-4692-b18b-6622e225c091 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:36.145049 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927586b-71f3-4692-b18b-6622e225c091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:36.145049 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927586b-71f3-4692-b18b-6622e225c091 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:36.145821 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3927586b-71f3-4692-b18b-6622e225c091 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:36.147877 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3927586b-71f3-4692-b18b-6622e225c091 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:36.161175 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1192/2373] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:21:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:21:36.165277 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab493122-b6c1-4dcd-9c17-ac3312987900 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:36.177510 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab493122-b6c1-4dcd-9c17-ac3312987900 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:36.183669 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4719a72b-08ff-4c27-85ec-423c44ef8c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:36.184598 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4719a72b-08ff-4c27-85ec-423c44ef8c80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:36.184880 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4719a72b-08ff-4c27-85ec-423c44ef8c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:36.185191 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4719a72b-08ff-4c27-85ec-423c44ef8c80 admin admin] PATH_INFO: `/v3/projects/8c2331d076b54458b5aa3b4fdd43f906/users/fe5cd41fe5f44fdf823948bc5ac487a8/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:36.201593 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4719a72b-08ff-4c27-85ec-423c44ef8c80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c2331d076b54458b5aa3b4fdd43f906, user_id=fe5cd41fe5f44fdf823948bc5ac487a8, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:36.204263 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4719a72b-08ff-4c27-85ec-423c44ef8c80 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:36.218546 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1182/2374] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:21:36 2022] PUT /identity/v3/projects/8c2331d076b54458b5aa3b4fdd43f906/users/fe5cd41fe5f44fdf823948bc5ac487a8/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:21:36.222766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927586b-71f3-4692-b18b-6622e225c091 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:36.235269 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927586b-71f3-4692-b18b-6622e225c091 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:36.300361 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f05cf-0f98-4f92-9345-bb3a18dbccb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:36.301071 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f05cf-0f98-4f92-9345-bb3a18dbccb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:36.301360 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f05cf-0f98-4f92-9345-bb3a18dbccb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:36.301633 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f05cf-0f98-4f92-9345-bb3a18dbccb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:36.302381 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9f05cf-0f98-4f92-9345-bb3a18dbccb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:36.304011 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9f05cf-0f98-4f92-9345-bb3a18dbccb6 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:36.316967 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1193/2375] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:21:36 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:21:36.320792 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4719a72b-08ff-4c27-85ec-423c44ef8c80 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:36.330333 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:36.334923 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4719a72b-08ff-4c27-85ec-423c44ef8c80 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:36.340765 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0c5ef8-9c15-4da7-b7ac-b5f195dc580e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:36.341865 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c5ef8-9c15-4da7-b7ac-b5f195dc580e admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:36.342149 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c5ef8-9c15-4da7-b7ac-b5f195dc580e admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:36.342420 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0c5ef8-9c15-4da7-b7ac-b5f195dc580e admin admin] PATH_INFO: `/v3/projects/8c2331d076b54458b5aa3b4fdd43f906/users/fe5cd41fe5f44fdf823948bc5ac487a8/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:36.344579 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0c5ef8-9c15-4da7-b7ac-b5f195dc580e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c2331d076b54458b5aa3b4fdd43f906, user_id=fe5cd41fe5f44fdf823948bc5ac487a8, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:36.347327 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0c5ef8-9c15-4da7-b7ac-b5f195dc580e admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:36.359167 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1183/2376] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:21:36 2022] PUT /identity/v3/projects/8c2331d076b54458b5aa3b4fdd43f906/users/fe5cd41fe5f44fdf823948bc5ac487a8/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:21:36.370876 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f535ed3-7e7d-461c-83e7-8e32e480c166 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:36.371176 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f535ed3-7e7d-461c-83e7-8e32e480c166 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:36.371427 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f535ed3-7e7d-461c-83e7-8e32e480c166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:36.407055 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-8f535ed3-7e7d-461c-83e7-8e32e480c166 None None] MFA Rules not processed for user `fe5cd41fe5f44fdf823948bc5ac487a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:21:36.408300 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-8f535ed3-7e7d-461c-83e7-8e32e480c166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:36.495570 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1194/2377] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:21:36 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:21:36.506573 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0c5ef8-9c15-4da7-b7ac-b5f195dc580e admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:36.509798 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-7a0c5ef8-9c15-4da7-b7ac-b5f195dc580e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:36.580211 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0c5ef8-9c15-4da7-b7ac-b5f195dc580e admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:36.584757 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd931a3-ae64-4069-9025-f7f91eb2fd7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:36.585470 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd931a3-ae64-4069-9025-f7f91eb2fd7e service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:36.585744 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd931a3-ae64-4069-9025-f7f91eb2fd7e service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:36.586019 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd931a3-ae64-4069-9025-f7f91eb2fd7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:36.668167 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd931a3-ae64-4069-9025-f7f91eb2fd7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:36.670737 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd931a3-ae64-4069-9025-f7f91eb2fd7e service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:36.683020 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1184/2378] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:21:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:21:37.547590 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f535ed3-7e7d-461c-83e7-8e32e480c166 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:37.549057 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-8f535ed3-7e7d-461c-83e7-8e32e480c166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:37.561500 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:37.613479 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f535ed3-7e7d-461c-83e7-8e32e480c166 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:37.617597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94989d0-42e3-45a2-ad47-f319bc091854 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:37.618339 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94989d0-42e3-45a2-ad47-f319bc091854 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:37.618767 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94989d0-42e3-45a2-ad47-f319bc091854 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:37.619039 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94989d0-42e3-45a2-ad47-f319bc091854 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:37.620962 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a94989d0-42e3-45a2-ad47-f319bc091854 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:37.634683 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94989d0-42e3-45a2-ad47-f319bc091854 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:37.637346 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94989d0-42e3-45a2-ad47-f319bc091854 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:37.649359 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1195/2379] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:21:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:21:39.306922 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd931a3-ae64-4069-9025-f7f91eb2fd7e service cinder] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:39.380348 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd931a3-ae64-4069-9025-f7f91eb2fd7e service cinder] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:39.387646 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb69d4-d622-4cae-87f8-d8a877ef78f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:39.388546 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb69d4-d622-4cae-87f8-d8a877ef78f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:39.388877 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb69d4-d622-4cae-87f8-d8a877ef78f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:39.389482 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb69d4-d622-4cae-87f8-d8a877ef78f9 admin admin] PATH_INFO: `/v3/users/a693560de0ab492eaf0766fe84b01acd` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:39.390594 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beb69d4-d622-4cae-87f8-d8a877ef78f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a693560de0ab492eaf0766fe84b01acd)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:39.393026 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beb69d4-d622-4cae-87f8-d8a877ef78f9 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:39.460487 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-1beb69d4-d622-4cae-87f8-d8a877ef78f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a693560de0ab492eaf0766fe84b01acd'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:39.462408 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:39.472841 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-1beb69d4-d622-4cae-87f8-d8a877ef78f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a693560de0ab492eaf0766fe84b01acd'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:39.473761 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-1beb69d4-d622-4cae-87f8-d8a877ef78f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a693560de0ab492eaf0766fe84b01acd'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:39.474934 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-1beb69d4-d622-4cae-87f8-d8a877ef78f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a693560de0ab492eaf0766fe84b01acd'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:39.490702 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-1beb69d4-d622-4cae-87f8-d8a877ef78f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a693560de0ab492eaf0766fe84b01acd'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:39.514914 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1185/2380] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:21:39 2022] DELETE /identity/v3/users/a693560de0ab492eaf0766fe84b01acd => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:21:39.519492 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8af89-3a7d-4a4f-b0f2-1613f6e63b11 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:39.520073 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8af89-3a7d-4a4f-b0f2-1613f6e63b11 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:39.520073 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8af89-3a7d-4a4f-b0f2-1613f6e63b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:39.538678 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-49d8af89-3a7d-4a4f-b0f2-1613f6e63b11 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:21:39.539969 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-49d8af89-3a7d-4a4f-b0f2-1613f6e63b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:39.608725 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1196/2381] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:21:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:21:39.619048 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb69d4-d622-4cae-87f8-d8a877ef78f9 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:39.620617 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-1beb69d4-d622-4cae-87f8-d8a877ef78f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:39.682743 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb69d4-d622-4cae-87f8-d8a877ef78f9 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:39.686953 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4896d7fc-de60-46c0-838e-571fcfaea429 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:39.687618 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896d7fc-de60-46c0-838e-571fcfaea429 service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:39.687903 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896d7fc-de60-46c0-838e-571fcfaea429 service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:39.688182 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896d7fc-de60-46c0-838e-571fcfaea429 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:39.699187 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4896d7fc-de60-46c0-838e-571fcfaea429 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:39.700689 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4896d7fc-de60-46c0-838e-571fcfaea429 service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:39.712492 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1186/2382] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:21:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:21:40.005974 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8af89-3a7d-4a4f-b0f2-1613f6e63b11 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:40.007546 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-49d8af89-3a7d-4a4f-b0f2-1613f6e63b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:40.024118 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8af89-3a7d-4a4f-b0f2-1613f6e63b11 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:40.043959 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3222d78-fd68-4ec6-9dc9-d2045259ffda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:40.044951 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3222d78-fd68-4ec6-9dc9-d2045259ffda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:40.045329 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3222d78-fd68-4ec6-9dc9-d2045259ffda admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:40.045531 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3222d78-fd68-4ec6-9dc9-d2045259ffda admin admin] PATH_INFO: `/v3/projects/ff00c4faa4754b2b8dcb1bad19806864` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:40.047132 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3222d78-fd68-4ec6-9dc9-d2045259ffda admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff00c4faa4754b2b8dcb1bad19806864)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:40.050299 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3222d78-fd68-4ec6-9dc9-d2045259ffda admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:40.104656 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c3222d78-fd68-4ec6-9dc9-d2045259ffda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff00c4faa4754b2b8dcb1bad19806864', 'request_id': 'req-c3222d78-fd68-4ec6-9dc9-d2045259ffda'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:40.111229 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c3222d78-fd68-4ec6-9dc9-d2045259ffda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff00c4faa4754b2b8dcb1bad19806864', 'request_id': 'req-c3222d78-fd68-4ec6-9dc9-d2045259ffda'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:40.119465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c3222d78-fd68-4ec6-9dc9-d2045259ffda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff00c4faa4754b2b8dcb1bad19806864', 'request_id': 'req-c3222d78-fd68-4ec6-9dc9-d2045259ffda'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:40.130397 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c3222d78-fd68-4ec6-9dc9-d2045259ffda admin admin] The token cache is being invalidate because project ff00c4faa4754b2b8dcb1bad19806864 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:21:40.130682 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c3222d78-fd68-4ec6-9dc9-d2045259ffda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:40.131311 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c3222d78-fd68-4ec6-9dc9-d2045259ffda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:40.132884 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1197/2383] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:21:40 2022] DELETE /identity/v3/projects/ff00c4faa4754b2b8dcb1bad19806864 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:21:40.137382 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4896d7fc-de60-46c0-838e-571fcfaea429 service neutron] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:40.147601 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4896d7fc-de60-46c0-838e-571fcfaea429 service neutron] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:40.214531 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c19f0-a977-47e0-95a6-9ad1a875c862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:40.215434 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c19f0-a977-47e0-95a6-9ad1a875c862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:40.215886 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c19f0-a977-47e0-95a6-9ad1a875c862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:40.216163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c19f0-a977-47e0-95a6-9ad1a875c862 admin admin] PATH_INFO: `/v3/users/fe5cd41fe5f44fdf823948bc5ac487a8` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:40.217521 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0c19f0-a977-47e0-95a6-9ad1a875c862 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe5cd41fe5f44fdf823948bc5ac487a8)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:40.219521 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0c19f0-a977-47e0-95a6-9ad1a875c862 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:40.271241 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-bb0c19f0-a977-47e0-95a6-9ad1a875c862 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe5cd41fe5f44fdf823948bc5ac487a8'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:40.279154 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-bb0c19f0-a977-47e0-95a6-9ad1a875c862 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe5cd41fe5f44fdf823948bc5ac487a8'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:40.280007 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-bb0c19f0-a977-47e0-95a6-9ad1a875c862 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe5cd41fe5f44fdf823948bc5ac487a8'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:40.280627 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-bb0c19f0-a977-47e0-95a6-9ad1a875c862 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe5cd41fe5f44fdf823948bc5ac487a8'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:40.289970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-bb0c19f0-a977-47e0-95a6-9ad1a875c862 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe5cd41fe5f44fdf823948bc5ac487a8'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:40.296015 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:40.311481 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1187/2384] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:21:40 2022] DELETE /identity/v3/users/fe5cd41fe5f44fdf823948bc5ac487a8 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:21:40.385510 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5edeb3-8536-43aa-a859-581f128f2601 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:40.385796 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5edeb3-8536-43aa-a859-581f128f2601 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:40.386070 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5edeb3-8536-43aa-a859-581f128f2601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:40.409392 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-0e5edeb3-8536-43aa-a859-581f128f2601 None None] MFA Rules not processed for user `f67110d591a94537a3df6a2d835370f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:21:40.410812 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-0e5edeb3-8536-43aa-a859-581f128f2601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:40.449156 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:40.488207 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1198/2385] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Jun 27 21:21:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:21:40.500173 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c19f0-a977-47e0-95a6-9ad1a875c862 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:40.502750 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-bb0c19f0-a977-47e0-95a6-9ad1a875c862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:40.592162 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c19f0-a977-47e0-95a6-9ad1a875c862 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:40.602722 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e7e605-eb9b-4910-858f-06488ab84f9a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bed6b231ee647d8a350da055d8e59d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:40.603522 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e7e605-eb9b-4910-858f-06488ab84f9a service swift] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:40.603806 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e7e605-eb9b-4910-858f-06488ab84f9a service swift] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:40.604079 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e7e605-eb9b-4910-858f-06488ab84f9a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:40.613822 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e7e605-eb9b-4910-858f-06488ab84f9a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:40.615865 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e7e605-eb9b-4910-858f-06488ab84f9a service swift] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:40.624884 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5edeb3-8536-43aa-a859-581f128f2601 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:40.627306 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-0e5edeb3-8536-43aa-a859-581f128f2601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:40.628570 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1188/2386] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Jun 27 21:21:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 21:21:40.707417 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5edeb3-8536-43aa-a859-581f128f2601 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:40.714721 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a14a6d1-6b64-4a8e-aa5d-17c78c13641a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:40.715722 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14a6d1-6b64-4a8e-aa5d-17c78c13641a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:40.716092 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14a6d1-6b64-4a8e-aa5d-17c78c13641a admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:40.716608 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14a6d1-6b64-4a8e-aa5d-17c78c13641a admin admin] PATH_INFO: `/v3/projects/8c2331d076b54458b5aa3b4fdd43f906` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:40.718730 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a14a6d1-6b64-4a8e-aa5d-17c78c13641a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c2331d076b54458b5aa3b4fdd43f906)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:40.722612 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a14a6d1-6b64-4a8e-aa5d-17c78c13641a admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:40.814733 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-2a14a6d1-6b64-4a8e-aa5d-17c78c13641a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c2331d076b54458b5aa3b4fdd43f906', 'request_id': 'req-2a14a6d1-6b64-4a8e-aa5d-17c78c13641a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:40.825169 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-2a14a6d1-6b64-4a8e-aa5d-17c78c13641a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c2331d076b54458b5aa3b4fdd43f906', 'request_id': 'req-2a14a6d1-6b64-4a8e-aa5d-17c78c13641a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:40.841412 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-2a14a6d1-6b64-4a8e-aa5d-17c78c13641a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c2331d076b54458b5aa3b4fdd43f906', 'request_id': 'req-2a14a6d1-6b64-4a8e-aa5d-17c78c13641a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:40.858709 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-2a14a6d1-6b64-4a8e-aa5d-17c78c13641a admin admin] The token cache is being invalidate because project 8c2331d076b54458b5aa3b4fdd43f906 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:21:40.859052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-2a14a6d1-6b64-4a8e-aa5d-17c78c13641a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:40.859833 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-2a14a6d1-6b64-4a8e-aa5d-17c78c13641a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:21:40.867013 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1199/2387] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:21:40 2022] DELETE /identity/v3/projects/8c2331d076b54458b5aa3b4fdd43f906 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:21:40.868887 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d29cb4-9670-43ff-a555-db6c9f571721 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:40.869314 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d29cb4-9670-43ff-a555-db6c9f571721 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:40.869696 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d29cb4-9670-43ff-a555-db6c9f571721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:40.890447 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-c9d29cb4-9670-43ff-a555-db6c9f571721 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:21:40.892132 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-c9d29cb4-9670-43ff-a555-db6c9f571721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:41.005018 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1189/2388] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:21:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:21:41.066808 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964daf04-8423-4af5-af67-4ad5d6fc097f None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:41.067393 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964daf04-8423-4af5-af67-4ad5d6fc097f None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:41.067826 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964daf04-8423-4af5-af67-4ad5d6fc097f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:41.088793 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-964daf04-8423-4af5-af67-4ad5d6fc097f None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:21:41.090322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-964daf04-8423-4af5-af67-4ad5d6fc097f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:41.103618 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d29cb4-9670-43ff-a555-db6c9f571721 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:41.110144 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1200/2389] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:21:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:21:41.114847 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d29cb4-9670-43ff-a555-db6c9f571721 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:41.120151 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6c4d77-d348-4c44-9b39-a4cdb9c91bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:41.121146 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c4d77-d348-4c44-9b39-a4cdb9c91bb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:41.121611 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c4d77-d348-4c44-9b39-a4cdb9c91bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:41.122100 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c4d77-d348-4c44-9b39-a4cdb9c91bb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:41.123012 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6c4d77-d348-4c44-9b39-a4cdb9c91bb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:21:41.123483 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6c4d77-d348-4c44-9b39-a4cdb9c91bb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:41.125310 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6c4d77-d348-4c44-9b39-a4cdb9c91bb9 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:41.154991 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1190/2390] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:21:41.156446 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964daf04-8423-4af5-af67-4ad5d6fc097f None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:41.167674 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964daf04-8423-4af5-af67-4ad5d6fc097f None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:41.172338 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e844b-2691-4fd8-a594-6edda628fb6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:41.173037 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e844b-2691-4fd8-a594-6edda628fb6b admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:41.173317 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e844b-2691-4fd8-a594-6edda628fb6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:41.173588 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e844b-2691-4fd8-a594-6edda628fb6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:41.174306 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315e844b-2691-4fd8-a594-6edda628fb6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:41.176236 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315e844b-2691-4fd8-a594-6edda628fb6b admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:41.209035 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1201/2391] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:21:41 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:21:41.214473 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6c4d77-d348-4c44-9b39-a4cdb9c91bb9 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:41.227357 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6c4d77-d348-4c44-9b39-a4cdb9c91bb9 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:41.288971 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c1ef9e-64d5-40af-9b61-fc1e141cbd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:41.289792 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1ef9e-64d5-40af-9b61-fc1e141cbd3b admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:41.290134 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1ef9e-64d5-40af-9b61-fc1e141cbd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:41.290418 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1ef9e-64d5-40af-9b61-fc1e141cbd3b admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:41.291420 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c1ef9e-64d5-40af-9b61-fc1e141cbd3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:41.293666 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c1ef9e-64d5-40af-9b61-fc1e141cbd3b admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:41.303030 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:41.321681 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1191/2392] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:21:41 2022] POST /identity/v3/users => generated 290 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:21:41.323687 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e844b-2691-4fd8-a594-6edda628fb6b admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:41.335602 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e844b-2691-4fd8-a594-6edda628fb6b admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:41.345220 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce322b9-68ea-4837-97e4-972211550d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:41.347236 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce322b9-68ea-4837-97e4-972211550d66 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:41.347236 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce322b9-68ea-4837-97e4-972211550d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:41.347456 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce322b9-68ea-4837-97e4-972211550d66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:41.349075 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce322b9-68ea-4837-97e4-972211550d66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:41.350769 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce322b9-68ea-4837-97e4-972211550d66 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:41.382593 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1202/2393] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:21:41.386603 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c1ef9e-64d5-40af-9b61-fc1e141cbd3b admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:41.408068 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c1ef9e-64d5-40af-9b61-fc1e141cbd3b admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:41.422495 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4565cef8-7ca3-4647-9802-c96c9e068fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:41.422854 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565cef8-7ca3-4647-9802-c96c9e068fee admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:41.423240 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565cef8-7ca3-4647-9802-c96c9e068fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:41.423612 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565cef8-7ca3-4647-9802-c96c9e068fee admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:41.424478 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4565cef8-7ca3-4647-9802-c96c9e068fee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:41.426283 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4565cef8-7ca3-4647-9802-c96c9e068fee admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:41.440536 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1192/2394] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:21:41.444711 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce322b9-68ea-4837-97e4-972211550d66 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:41.456382 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce322b9-68ea-4837-97e4-972211550d66 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:41.462296 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa45f3ab-faff-43af-b637-ba82bc3cc162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:41.463411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45f3ab-faff-43af-b637-ba82bc3cc162 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:41.463772 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45f3ab-faff-43af-b637-ba82bc3cc162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:41.464101 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45f3ab-faff-43af-b637-ba82bc3cc162 admin admin] PATH_INFO: `/v3/projects/824cb65198bf417fbc4baeae349d358d/users/e4b46dc825704fbc9cc2505805f22f94/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:41.487058 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa45f3ab-faff-43af-b637-ba82bc3cc162 admin admin] RBAC: Authorizing `identity:create_grant(project_id=824cb65198bf417fbc4baeae349d358d, user_id=e4b46dc825704fbc9cc2505805f22f94, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:41.490520 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa45f3ab-faff-43af-b637-ba82bc3cc162 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:41.511560 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1203/2395] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:21:41 2022] PUT /identity/v3/projects/824cb65198bf417fbc4baeae349d358d/users/e4b46dc825704fbc9cc2505805f22f94/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:21:41.516613 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7ec1f-eb2a-4d9b-9a69-c2d1618c93f4 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:41.516958 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7ec1f-eb2a-4d9b-9a69-c2d1618c93f4 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:41.517397 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7ec1f-eb2a-4d9b-9a69-c2d1618c93f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:41.541489 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-37e7ec1f-eb2a-4d9b-9a69-c2d1618c93f4 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:21:41.542807 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-37e7ec1f-eb2a-4d9b-9a69-c2d1618c93f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:41.631777 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1193/2396] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:21:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:21:41.642289 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa45f3ab-faff-43af-b637-ba82bc3cc162 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:41.644274 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-fa45f3ab-faff-43af-b637-ba82bc3cc162 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:41.654801 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:41.733275 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa45f3ab-faff-43af-b637-ba82bc3cc162 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:41.737126 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851292c-621c-411c-948f-636a734566cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:41.737727 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851292c-621c-411c-948f-636a734566cc service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:41.738008 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851292c-621c-411c-948f-636a734566cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:41.738281 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851292c-621c-411c-948f-636a734566cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:41.748469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6851292c-621c-411c-948f-636a734566cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:41.749959 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6851292c-621c-411c-948f-636a734566cc service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:41.776050 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1204/2397] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:21:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:21:47.136509 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e939f9f9-3dec-431d-8ed8-eeb5adfc141c None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:47.137466 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e939f9f9-3dec-431d-8ed8-eeb5adfc141c None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:47.137466 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e939f9f9-3dec-431d-8ed8-eeb5adfc141c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:47.178841 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-e939f9f9-3dec-431d-8ed8-eeb5adfc141c None None] MFA Rules not processed for user `e4b46dc825704fbc9cc2505805f22f94`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:21:47.180341 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-e939f9f9-3dec-431d-8ed8-eeb5adfc141c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:47.279135 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1194/2398] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:21:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:21:47.285674 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851292c-621c-411c-948f-636a734566cc service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:47.304283 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851292c-621c-411c-948f-636a734566cc service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:47.312817 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efeb7ad-34e9-4506-8b59-0ba5828ec1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:47.313252 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efeb7ad-34e9-4506-8b59-0ba5828ec1cb admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:47.313846 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efeb7ad-34e9-4506-8b59-0ba5828ec1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:47.314117 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efeb7ad-34e9-4506-8b59-0ba5828ec1cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:47.314831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efeb7ad-34e9-4506-8b59-0ba5828ec1cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:47.317690 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efeb7ad-34e9-4506-8b59-0ba5828ec1cb admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:47.373356 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1205/2399] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:21:47 2022] POST /identity/v3/projects => generated 379 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:21:47.374633 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e939f9f9-3dec-431d-8ed8-eeb5adfc141c None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:47.416260 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e939f9f9-3dec-431d-8ed8-eeb5adfc141c None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:47.513836 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c4ee32-3293-4374-a9f1-38949851ff68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:47.514692 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4ee32-3293-4374-a9f1-38949851ff68 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:47.515030 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4ee32-3293-4374-a9f1-38949851ff68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:47.515367 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c4ee32-3293-4374-a9f1-38949851ff68 admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:47.516151 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c4ee32-3293-4374-a9f1-38949851ff68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:47.518289 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c4ee32-3293-4374-a9f1-38949851ff68 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:47.566335 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1195/2400] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:21:47 2022] POST /identity/v3/users => generated 296 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:21:47.573798 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efeb7ad-34e9-4506-8b59-0ba5828ec1cb admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:47.589119 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efeb7ad-34e9-4506-8b59-0ba5828ec1cb admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:47.604336 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f3cb6-58fa-44e5-b943-b8d500468f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:47.605300 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f3cb6-58fa-44e5-b943-b8d500468f8d admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:47.605546 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f3cb6-58fa-44e5-b943-b8d500468f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:47.605828 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f3cb6-58fa-44e5-b943-b8d500468f8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:47.606724 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42f3cb6-58fa-44e5-b943-b8d500468f8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:47.609129 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42f3cb6-58fa-44e5-b943-b8d500468f8d admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:47.621444 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1206/2401] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:21:47 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:21:47.625743 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c4ee32-3293-4374-a9f1-38949851ff68 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:47.635872 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:47.651311 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c4ee32-3293-4374-a9f1-38949851ff68 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:47.669206 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00ba2c0-1346-49a3-afe8-bab6eefc9e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:47.670199 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ba2c0-1346-49a3-afe8-bab6eefc9e4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:47.670485 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ba2c0-1346-49a3-afe8-bab6eefc9e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:47.670761 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ba2c0-1346-49a3-afe8-bab6eefc9e4d admin admin] PATH_INFO: `/v3/projects/3a7fee95fa2a436bbe9c4ca355a9a1a6/users/4640e362bf7349deb51ab58a376091e5/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:47.691350 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00ba2c0-1346-49a3-afe8-bab6eefc9e4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a7fee95fa2a436bbe9c4ca355a9a1a6, user_id=4640e362bf7349deb51ab58a376091e5, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:47.694861 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00ba2c0-1346-49a3-afe8-bab6eefc9e4d admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:47.716978 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1196/2402] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:21:47 2022] PUT /identity/v3/projects/3a7fee95fa2a436bbe9c4ca355a9a1a6/users/4640e362bf7349deb51ab58a376091e5/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:21:47.720228 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f3cb6-58fa-44e5-b943-b8d500468f8d admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:47.736622 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f3cb6-58fa-44e5-b943-b8d500468f8d admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:47.835874 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd4698-ea7b-4a9a-b83c-a0c50a725cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:47.836766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd4698-ea7b-4a9a-b83c-a0c50a725cfb admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:47.837278 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd4698-ea7b-4a9a-b83c-a0c50a725cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:47.837627 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd4698-ea7b-4a9a-b83c-a0c50a725cfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:47.838640 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd4698-ea7b-4a9a-b83c-a0c50a725cfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:47.840603 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd4698-ea7b-4a9a-b83c-a0c50a725cfb admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:47.855262 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1207/2403] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:21:47 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:21:47.859372 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00ba2c0-1346-49a3-afe8-bab6eefc9e4d admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:47.871259 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00ba2c0-1346-49a3-afe8-bab6eefc9e4d admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:47.876407 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcae288-cfff-4a34-ac07-2859981a7b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:47.877648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcae288-cfff-4a34-ac07-2859981a7b44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:47.877648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcae288-cfff-4a34-ac07-2859981a7b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:47.877895 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcae288-cfff-4a34-ac07-2859981a7b44 admin admin] PATH_INFO: `/v3/projects/3a7fee95fa2a436bbe9c4ca355a9a1a6/users/4640e362bf7349deb51ab58a376091e5/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:47.879996 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcae288-cfff-4a34-ac07-2859981a7b44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a7fee95fa2a436bbe9c4ca355a9a1a6, user_id=4640e362bf7349deb51ab58a376091e5, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:47.883752 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcae288-cfff-4a34-ac07-2859981a7b44 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:47.903062 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1197/2404] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:21:47 2022] PUT /identity/v3/projects/3a7fee95fa2a436bbe9c4ca355a9a1a6/users/4640e362bf7349deb51ab58a376091e5/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:21:48.807163 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd4698-ea7b-4a9a-b83c-a0c50a725cfb admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:48.809064 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-4ccd4698-ea7b-4a9a-b83c-a0c50a725cfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:48.933194 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd4698-ea7b-4a9a-b83c-a0c50a725cfb admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:48.982577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:21:49.134690 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ed6f03-a045-4318-9afd-4de3d0497ffa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:49.135578 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed6f03-a045-4318-9afd-4de3d0497ffa service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:49.136463 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed6f03-a045-4318-9afd-4de3d0497ffa service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:49.137813 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed6f03-a045-4318-9afd-4de3d0497ffa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:49.148203 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-99ed6f03-a045-4318-9afd-4de3d0497ffa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:49.284599 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ed6f03-a045-4318-9afd-4de3d0497ffa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:49.286583 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ed6f03-a045-4318-9afd-4de3d0497ffa service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:49.302246 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1208/2405] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:21:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:21:54.528220 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83730c1c-6f6b-4eeb-8928-a64670f813ea None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:54.529473 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83730c1c-6f6b-4eeb-8928-a64670f813ea None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:54.529846 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83730c1c-6f6b-4eeb-8928-a64670f813ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:54.562909 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-83730c1c-6f6b-4eeb-8928-a64670f813ea None None] MFA Rules not processed for user `4640e362bf7349deb51ab58a376091e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:21:54.564766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-83730c1c-6f6b-4eeb-8928-a64670f813ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:54.654156 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1198/2406] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:21:54 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:21:54.670544 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ed6f03-a045-4318-9afd-4de3d0497ffa service nova] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:54.684629 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ed6f03-a045-4318-9afd-4de3d0497ffa service nova] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:54.693116 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70156d2a-f0d5-46bc-ba38-47062837ae9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:54.694025 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70156d2a-f0d5-46bc-ba38-47062837ae9b service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:54.694443 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70156d2a-f0d5-46bc-ba38-47062837ae9b service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:54.695057 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70156d2a-f0d5-46bc-ba38-47062837ae9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:54.766695 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70156d2a-f0d5-46bc-ba38-47062837ae9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:54.768350 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70156d2a-f0d5-46bc-ba38-47062837ae9b service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:54.780885 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1209/2407] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:21:54 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:21:58.130645 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83730c1c-6f6b-4eeb-8928-a64670f813ea None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:58.147223 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83730c1c-6f6b-4eeb-8928-a64670f813ea None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:58.168342 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97b740d-f914-4584-9cfd-8da63a1e220d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:58.170059 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b740d-f914-4584-9cfd-8da63a1e220d service nova] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:58.170545 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b740d-f914-4584-9cfd-8da63a1e220d service nova] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:58.172108 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b740d-f914-4584-9cfd-8da63a1e220d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:58.177829 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-a97b740d-f914-4584-9cfd-8da63a1e220d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:58.228765 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97b740d-f914-4584-9cfd-8da63a1e220d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:58.230438 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97b740d-f914-4584-9cfd-8da63a1e220d service nova] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:58.251563 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1199/2408] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:21:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:21:59.282952 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70156d2a-f0d5-46bc-ba38-47062837ae9b service nova] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:21:59.285630 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-70156d2a-f0d5-46bc-ba38-47062837ae9b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:59.307155 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70156d2a-f0d5-46bc-ba38-47062837ae9b service nova] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:21:59.313798 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef631653-0570-40cd-b608-1438b03d939f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:21:59.314665 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef631653-0570-40cd-b608-1438b03d939f service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:21:59.314952 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef631653-0570-40cd-b608-1438b03d939f service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:21:59.315228 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef631653-0570-40cd-b608-1438b03d939f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:21:59.317563 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ef631653-0570-40cd-b608-1438b03d939f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:21:59.347148 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef631653-0570-40cd-b608-1438b03d939f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:21:59.349848 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef631653-0570-40cd-b608-1438b03d939f service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:21:59.376318 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1210/2409] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:21:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:22:00.347754 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38620010-54c2-4d03-a935-c25e239e25ed None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:00.348297 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38620010-54c2-4d03-a935-c25e239e25ed None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:00.348297 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38620010-54c2-4d03-a935-c25e239e25ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:00.366943 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-38620010-54c2-4d03-a935-c25e239e25ed None None] MFA Rules not processed for user `f67110d591a94537a3df6a2d835370f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:00.368558 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-38620010-54c2-4d03-a935-c25e239e25ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:00.454521 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1200/2410] 10.222.0.13 () {58 vars in 967 bytes} [Mon Jun 27 21:22:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:00.469029 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef631653-0570-40cd-b608-1438b03d939f service nova] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:00.470410 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ef631653-0570-40cd-b608-1438b03d939f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:00.511489 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:00.554925 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef631653-0570-40cd-b608-1438b03d939f service nova] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:00.561322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ba9b2-f042-433c-8ef4-cdbbfd44aa02 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bed6b231ee647d8a350da055d8e59d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:00.562020 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ba9b2-f042-433c-8ef4-cdbbfd44aa02 service swift] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:00.562353 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ba9b2-f042-433c-8ef4-cdbbfd44aa02 service swift] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:00.562627 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ba9b2-f042-433c-8ef4-cdbbfd44aa02 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:00.572061 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5ba9b2-f042-433c-8ef4-cdbbfd44aa02 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:00.573734 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5ba9b2-f042-433c-8ef4-cdbbfd44aa02 service swift] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:00.583846 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1211/2411] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Jun 27 21:22:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 21:22:06.669702 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38620010-54c2-4d03-a935-c25e239e25ed None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:06.755710 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:06.772383 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38620010-54c2-4d03-a935-c25e239e25ed None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:06.777080 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ccde13-1d43-4028-8934-253219b020bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:06.777759 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ccde13-1d43-4028-8934-253219b020bb service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:06.779335 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ccde13-1d43-4028-8934-253219b020bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:06.779335 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ccde13-1d43-4028-8934-253219b020bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:06.780476 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-a3ccde13-1d43-4028-8934-253219b020bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:06.866551 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ccde13-1d43-4028-8934-253219b020bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:06.868455 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ccde13-1d43-4028-8934-253219b020bb service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:06.881015 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1201/2412] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:22:10.157239 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ba9b2-f042-433c-8ef4-cdbbfd44aa02 service swift] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:10.158879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-9f5ba9b2-f042-433c-8ef4-cdbbfd44aa02 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:10.173424 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ba9b2-f042-433c-8ef4-cdbbfd44aa02 service swift] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:10.178136 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda7a29-d144-4d91-a764-597e2745b289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:10.179021 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda7a29-d144-4d91-a764-597e2745b289 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:10.179419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda7a29-d144-4d91-a764-597e2745b289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:10.179870 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda7a29-d144-4d91-a764-597e2745b289 admin admin] PATH_INFO: `/v3/users/7969c6620a0d406890fb4c15b58ab349` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:10.181360 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda7a29-d144-4d91-a764-597e2745b289 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7969c6620a0d406890fb4c15b58ab349)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:10.183315 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda7a29-d144-4d91-a764-597e2745b289 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:10.234268 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-adda7a29-d144-4d91-a764-597e2745b289 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7969c6620a0d406890fb4c15b58ab349'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:10.241326 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-adda7a29-d144-4d91-a764-597e2745b289 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7969c6620a0d406890fb4c15b58ab349'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:10.242088 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-adda7a29-d144-4d91-a764-597e2745b289 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7969c6620a0d406890fb4c15b58ab349'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:10.242797 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-adda7a29-d144-4d91-a764-597e2745b289 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7969c6620a0d406890fb4c15b58ab349'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:10.250763 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-adda7a29-d144-4d91-a764-597e2745b289 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7969c6620a0d406890fb4c15b58ab349'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:10.268572 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1212/2413] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:22:10 2022] DELETE /identity/v3/users/7969c6620a0d406890fb4c15b58ab349 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:10.476556 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ccde13-1d43-4028-8934-253219b020bb service neutron] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:10.495981 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ccde13-1d43-4028-8934-253219b020bb service neutron] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:10.559340 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb17140-b624-4283-a185-92845a97564c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:10.560224 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb17140-b624-4283-a185-92845a97564c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:10.560529 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb17140-b624-4283-a185-92845a97564c admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:10.560799 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb17140-b624-4283-a185-92845a97564c admin admin] PATH_INFO: `/v3/projects/d8c1bcb797ba4eb0ad6daf5ed39baa89` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:10.562063 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb17140-b624-4283-a185-92845a97564c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8c1bcb797ba4eb0ad6daf5ed39baa89)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:10.563956 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb17140-b624-4283-a185-92845a97564c admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:10.613264 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-fdb17140-b624-4283-a185-92845a97564c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8c1bcb797ba4eb0ad6daf5ed39baa89', 'request_id': 'req-fdb17140-b624-4283-a185-92845a97564c'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:10.626651 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-fdb17140-b624-4283-a185-92845a97564c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8c1bcb797ba4eb0ad6daf5ed39baa89', 'request_id': 'req-fdb17140-b624-4283-a185-92845a97564c'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:10.636567 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-fdb17140-b624-4283-a185-92845a97564c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8c1bcb797ba4eb0ad6daf5ed39baa89', 'request_id': 'req-fdb17140-b624-4283-a185-92845a97564c'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:10.649140 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-fdb17140-b624-4283-a185-92845a97564c admin admin] The token cache is being invalidate because project d8c1bcb797ba4eb0ad6daf5ed39baa89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:22:10.649437 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-fdb17140-b624-4283-a185-92845a97564c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:10.650120 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-fdb17140-b624-4283-a185-92845a97564c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:10.652181 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1202/2414] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:22:10 2022] DELETE /identity/v3/projects/d8c1bcb797ba4eb0ad6daf5ed39baa89 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:10.657440 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a823e-b7ae-4c3a-a1be-65ec577c1fb6 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:10.658206 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a823e-b7ae-4c3a-a1be-65ec577c1fb6 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:10.658206 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a823e-b7ae-4c3a-a1be-65ec577c1fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:10.685607 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-971a823e-b7ae-4c3a-a1be-65ec577c1fb6 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:10.686914 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-971a823e-b7ae-4c3a-a1be-65ec577c1fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:10.768952 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1213/2415] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:22:10.770697 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfec07f-e6e1-45bf-9dcb-1894fa6095da None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:10.770990 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfec07f-e6e1-45bf-9dcb-1894fa6095da None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:10.771272 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfec07f-e6e1-45bf-9dcb-1894fa6095da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:10.791060 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-3bfec07f-e6e1-45bf-9dcb-1894fa6095da None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:10.792273 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-3bfec07f-e6e1-45bf-9dcb-1894fa6095da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:10.800518 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1203/2416] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:22:10.805215 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a823e-b7ae-4c3a-a1be-65ec577c1fb6 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:10.815554 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a823e-b7ae-4c3a-a1be-65ec577c1fb6 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:10.820630 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81637f-a3f7-4f6c-89b5-f996e83db67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:10.821416 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81637f-a3f7-4f6c-89b5-f996e83db67f admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:10.821690 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81637f-a3f7-4f6c-89b5-f996e83db67f admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:10.821962 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81637f-a3f7-4f6c-89b5-f996e83db67f admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:10.822704 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef81637f-a3f7-4f6c-89b5-f996e83db67f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:22:10.823014 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef81637f-a3f7-4f6c-89b5-f996e83db67f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:10.824641 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef81637f-a3f7-4f6c-89b5-f996e83db67f admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:10.839019 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1214/2417] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:22:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:22:10.843683 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfec07f-e6e1-45bf-9dcb-1894fa6095da None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:10.855619 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfec07f-e6e1-45bf-9dcb-1894fa6095da None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:10.861498 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43b491-d0ca-40f0-bb27-ab4777b78d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:10.862180 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43b491-d0ca-40f0-bb27-ab4777b78d0d admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:10.862469 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43b491-d0ca-40f0-bb27-ab4777b78d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:10.862746 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43b491-d0ca-40f0-bb27-ab4777b78d0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:10.863470 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba43b491-d0ca-40f0-bb27-ab4777b78d0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:10.865414 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba43b491-d0ca-40f0-bb27-ab4777b78d0d admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:10.881534 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:10.909095 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1204/2418] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:22:10 2022] POST /identity/v3/projects => generated 385 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:22:10.916389 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81637f-a3f7-4f6c-89b5-f996e83db67f admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:10.927179 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81637f-a3f7-4f6c-89b5-f996e83db67f admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:10.951845 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:11.003241 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0eb1f-1766-472b-8bc7-ff7909830c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:11.004000 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0eb1f-1766-472b-8bc7-ff7909830c3b admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:11.004253 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0eb1f-1766-472b-8bc7-ff7909830c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:11.004539 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0eb1f-1766-472b-8bc7-ff7909830c3b admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:11.005291 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a0eb1f-1766-472b-8bc7-ff7909830c3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:11.007269 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a0eb1f-1766-472b-8bc7-ff7909830c3b admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:11.031991 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1215/2419] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:22:10 2022] POST /identity/v3/users => generated 293 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:22:11.037049 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43b491-d0ca-40f0-bb27-ab4777b78d0d admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:11.049898 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43b491-d0ca-40f0-bb27-ab4777b78d0d admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:11.054775 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60fe703-1a30-4cdb-ad74-0e65f05a7c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:11.055646 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fe703-1a30-4cdb-ad74-0e65f05a7c49 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:11.055929 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fe703-1a30-4cdb-ad74-0e65f05a7c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:11.056203 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fe703-1a30-4cdb-ad74-0e65f05a7c49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:11.057001 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60fe703-1a30-4cdb-ad74-0e65f05a7c49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:11.058906 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60fe703-1a30-4cdb-ad74-0e65f05a7c49 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:11.072698 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1205/2420] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:22:11.077102 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0eb1f-1766-472b-8bc7-ff7909830c3b admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:11.090657 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0eb1f-1766-472b-8bc7-ff7909830c3b admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:11.096304 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aa28f5-daa6-4268-9367-2bb7ada90ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:11.096992 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa28f5-daa6-4268-9367-2bb7ada90ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:11.097283 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa28f5-daa6-4268-9367-2bb7ada90ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:11.097579 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa28f5-daa6-4268-9367-2bb7ada90ce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:11.098340 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aa28f5-daa6-4268-9367-2bb7ada90ce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:11.099967 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aa28f5-daa6-4268-9367-2bb7ada90ce2 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:11.113567 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1216/2421] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:22:11.118838 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60fe703-1a30-4cdb-ad74-0e65f05a7c49 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:11.129840 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60fe703-1a30-4cdb-ad74-0e65f05a7c49 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:11.135315 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce738d8-dc86-426f-bf53-036812c9c804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:11.136239 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce738d8-dc86-426f-bf53-036812c9c804 admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:11.136523 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce738d8-dc86-426f-bf53-036812c9c804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:11.136800 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce738d8-dc86-426f-bf53-036812c9c804 admin admin] PATH_INFO: `/v3/projects/491ea0fd07644de68f0110114edf9b26/users/1c6dc1c8e77449e693e2b70d586b7949/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:11.153838 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce738d8-dc86-426f-bf53-036812c9c804 admin admin] RBAC: Authorizing `identity:create_grant(project_id=491ea0fd07644de68f0110114edf9b26, user_id=1c6dc1c8e77449e693e2b70d586b7949, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:11.156625 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce738d8-dc86-426f-bf53-036812c9c804 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:11.172775 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1206/2422] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:22:11 2022] PUT /identity/v3/projects/491ea0fd07644de68f0110114edf9b26/users/1c6dc1c8e77449e693e2b70d586b7949/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:11.185236 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73220ad-b04f-4cd3-9d00-17790128d7b3 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:11.185873 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73220ad-b04f-4cd3-9d00-17790128d7b3 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:11.185873 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73220ad-b04f-4cd3-9d00-17790128d7b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:11.218005 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-c73220ad-b04f-4cd3-9d00-17790128d7b3 None None] MFA Rules not processed for user `1c6dc1c8e77449e693e2b70d586b7949`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:11.219290 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-c73220ad-b04f-4cd3-9d00-17790128d7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:11.290642 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1217/2423] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:11.295370 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce738d8-dc86-426f-bf53-036812c9c804 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:11.360402 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce738d8-dc86-426f-bf53-036812c9c804 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:11.365556 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ecdb19-1640-4562-8f16-5b04b5665dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:11.366235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ecdb19-1640-4562-8f16-5b04b5665dae admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:11.366554 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ecdb19-1640-4562-8f16-5b04b5665dae admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:11.366799 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ecdb19-1640-4562-8f16-5b04b5665dae admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:11.367628 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ecdb19-1640-4562-8f16-5b04b5665dae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:11.369586 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ecdb19-1640-4562-8f16-5b04b5665dae admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:11.398280 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1207/2424] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:22:11 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:22:11.402608 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73220ad-b04f-4cd3-9d00-17790128d7b3 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:11.413857 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73220ad-b04f-4cd3-9d00-17790128d7b3 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:11.475333 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2d73fe-b6a8-4961-a4ae-ccb0cf146827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:11.475916 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d73fe-b6a8-4961-a4ae-ccb0cf146827 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:11.476412 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d73fe-b6a8-4961-a4ae-ccb0cf146827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:11.476532 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d73fe-b6a8-4961-a4ae-ccb0cf146827 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:11.477250 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2d73fe-b6a8-4961-a4ae-ccb0cf146827 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:11.479213 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2d73fe-b6a8-4961-a4ae-ccb0cf146827 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:11.499015 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1218/2425] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:22:11 2022] POST /identity/v3/users => generated 300 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:22:11.503649 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ecdb19-1640-4562-8f16-5b04b5665dae admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:11.514676 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ecdb19-1640-4562-8f16-5b04b5665dae admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:11.519578 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3122364d-b7f1-4194-8425-6bd9649efad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:11.520249 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122364d-b7f1-4194-8425-6bd9649efad7 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:11.520534 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122364d-b7f1-4194-8425-6bd9649efad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:11.520808 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122364d-b7f1-4194-8425-6bd9649efad7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:11.521581 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3122364d-b7f1-4194-8425-6bd9649efad7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:11.523258 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3122364d-b7f1-4194-8425-6bd9649efad7 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:11.534515 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1208/2426] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:22:11.539289 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2d73fe-b6a8-4961-a4ae-ccb0cf146827 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:11.549906 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2d73fe-b6a8-4961-a4ae-ccb0cf146827 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:11.554891 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8da952-7b75-4815-917a-f2e71c0c72c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:11.555790 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8da952-7b75-4815-917a-f2e71c0c72c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:11.556081 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8da952-7b75-4815-917a-f2e71c0c72c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:11.556362 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8da952-7b75-4815-917a-f2e71c0c72c7 admin admin] PATH_INFO: `/v3/projects/1f2f8b0b514f41ed9a204f2587f9f503/users/04c7bae8cb104db2b613d515f9454ca3/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:11.573405 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8da952-7b75-4815-917a-f2e71c0c72c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f2f8b0b514f41ed9a204f2587f9f503, user_id=04c7bae8cb104db2b613d515f9454ca3, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:11.576123 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8da952-7b75-4815-917a-f2e71c0c72c7 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:11.579793 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:11.593790 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1219/2427] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:22:11 2022] PUT /identity/v3/projects/1f2f8b0b514f41ed9a204f2587f9f503/users/04c7bae8cb104db2b613d515f9454ca3/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:22:11.597856 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3122364d-b7f1-4194-8425-6bd9649efad7 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:11.608296 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3122364d-b7f1-4194-8425-6bd9649efad7 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:11.667923 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e359c8c-2c36-4295-9f82-1d007bb6634e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:11.668599 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e359c8c-2c36-4295-9f82-1d007bb6634e admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:11.668877 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e359c8c-2c36-4295-9f82-1d007bb6634e admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:11.669183 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e359c8c-2c36-4295-9f82-1d007bb6634e admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:11.669907 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e359c8c-2c36-4295-9f82-1d007bb6634e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:11.671511 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e359c8c-2c36-4295-9f82-1d007bb6634e admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:11.684269 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1209/2428] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:11 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:22:11.691716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8da952-7b75-4815-917a-f2e71c0c72c7 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:11.708285 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8da952-7b75-4815-917a-f2e71c0c72c7 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:11.714800 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aefe4d-cf9e-404c-8323-b1b82d9fd3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:11.715976 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aefe4d-cf9e-404c-8323-b1b82d9fd3ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:11.716222 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aefe4d-cf9e-404c-8323-b1b82d9fd3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:11.716482 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aefe4d-cf9e-404c-8323-b1b82d9fd3ea admin admin] PATH_INFO: `/v3/projects/1f2f8b0b514f41ed9a204f2587f9f503/users/04c7bae8cb104db2b613d515f9454ca3/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:11.718760 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aefe4d-cf9e-404c-8323-b1b82d9fd3ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f2f8b0b514f41ed9a204f2587f9f503, user_id=04c7bae8cb104db2b613d515f9454ca3, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:11.721522 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aefe4d-cf9e-404c-8323-b1b82d9fd3ea admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:11.733722 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1220/2429] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:22:11 2022] PUT /identity/v3/projects/1f2f8b0b514f41ed9a204f2587f9f503/users/04c7bae8cb104db2b613d515f9454ca3/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:11.744371 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a89d87-358e-48fa-8a6d-3f639eab1cb8 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:11.744653 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a89d87-358e-48fa-8a6d-3f639eab1cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:11.744949 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a89d87-358e-48fa-8a6d-3f639eab1cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:11.776490 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-22a89d87-358e-48fa-8a6d-3f639eab1cb8 None None] MFA Rules not processed for user `04c7bae8cb104db2b613d515f9454ca3`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:11.777753 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-22a89d87-358e-48fa-8a6d-3f639eab1cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:11.787075 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:11.865880 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1210/2430] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:11 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:11.882437 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aefe4d-cf9e-404c-8323-b1b82d9fd3ea admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:11.883999 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a89d87-358e-48fa-8a6d-3f639eab1cb8 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:11.884474 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-c9aefe4d-cf9e-404c-8323-b1b82d9fd3ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:11.885581 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-22a89d87-358e-48fa-8a6d-3f639eab1cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:11.950519 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a89d87-358e-48fa-8a6d-3f639eab1cb8 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:11.955235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb2553-a291-4747-9103-d375c8e5e024 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:11.955901 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb2553-a291-4747-9103-d375c8e5e024 service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:11.956192 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb2553-a291-4747-9103-d375c8e5e024 service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:11.956468 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb2553-a291-4747-9103-d375c8e5e024 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:11.958871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-40bb2553-a291-4747-9103-d375c8e5e024 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:11.969840 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aefe4d-cf9e-404c-8323-b1b82d9fd3ea admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:11.974548 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a20bc-6bde-4c24-a6d6-ace0f26f726d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:11.975370 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a20bc-6bde-4c24-a6d6-ace0f26f726d service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:11.975703 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a20bc-6bde-4c24-a6d6-ace0f26f726d service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:11.975983 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a20bc-6bde-4c24-a6d6-ace0f26f726d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:11.987123 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a20bc-6bde-4c24-a6d6-ace0f26f726d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:11.989135 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a20bc-6bde-4c24-a6d6-ace0f26f726d service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:11.999937 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1221/2431] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:11 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:22:12.037988 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bb2553-a291-4747-9103-d375c8e5e024 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:12.043981 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bb2553-a291-4747-9103-d375c8e5e024 service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:12.055650 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1211/2432] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:22:12.073748 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a20bc-6bde-4c24-a6d6-ace0f26f726d service cinder] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:12.085883 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a20bc-6bde-4c24-a6d6-ace0f26f726d service cinder] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:12.090869 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab3f3ac-fc85-4131-97f3-771e4ff8b4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:12.091755 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab3f3ac-fc85-4131-97f3-771e4ff8b4d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:12.092038 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab3f3ac-fc85-4131-97f3-771e4ff8b4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:12.092309 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab3f3ac-fc85-4131-97f3-771e4ff8b4d8 admin admin] PATH_INFO: `/v3/users/1c6dc1c8e77449e693e2b70d586b7949` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:12.093561 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab3f3ac-fc85-4131-97f3-771e4ff8b4d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c6dc1c8e77449e693e2b70d586b7949)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:12.095502 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab3f3ac-fc85-4131-97f3-771e4ff8b4d8 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:12.164675 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-dab3f3ac-fc85-4131-97f3-771e4ff8b4d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c6dc1c8e77449e693e2b70d586b7949'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:12.174197 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-dab3f3ac-fc85-4131-97f3-771e4ff8b4d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c6dc1c8e77449e693e2b70d586b7949'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:12.175104 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-dab3f3ac-fc85-4131-97f3-771e4ff8b4d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c6dc1c8e77449e693e2b70d586b7949'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:12.175972 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-dab3f3ac-fc85-4131-97f3-771e4ff8b4d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c6dc1c8e77449e693e2b70d586b7949'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:12.186954 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-dab3f3ac-fc85-4131-97f3-771e4ff8b4d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c6dc1c8e77449e693e2b70d586b7949'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:12.213020 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1222/2433] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:22:12 2022] DELETE /identity/v3/users/1c6dc1c8e77449e693e2b70d586b7949 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:22:12.217807 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72915d5-9867-42e0-af76-f0f604a2d051 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:12.218172 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72915d5-9867-42e0-af76-f0f604a2d051 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:12.218511 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72915d5-9867-42e0-af76-f0f604a2d051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:12.238353 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-f72915d5-9867-42e0-af76-f0f604a2d051 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:12.239682 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-f72915d5-9867-42e0-af76-f0f604a2d051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:12.298787 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:12.312561 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1212/2434] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:12.322493 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab3f3ac-fc85-4131-97f3-771e4ff8b4d8 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:12.324030 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-dab3f3ac-fc85-4131-97f3-771e4ff8b4d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:12.391464 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab3f3ac-fc85-4131-97f3-771e4ff8b4d8 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:12.395029 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8877a55f-cc89-4d24-9466-bc13b841c5b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:12.395705 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877a55f-cc89-4d24-9466-bc13b841c5b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:12.395982 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877a55f-cc89-4d24-9466-bc13b841c5b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:12.396253 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877a55f-cc89-4d24-9466-bc13b841c5b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:12.406687 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8877a55f-cc89-4d24-9466-bc13b841c5b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:12.408130 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8877a55f-cc89-4d24-9466-bc13b841c5b4 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:12.413186 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:12.419346 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1223/2435] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:22:12.746602 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72915d5-9867-42e0-af76-f0f604a2d051 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:12.748273 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-f72915d5-9867-42e0-af76-f0f604a2d051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:12.768131 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72915d5-9867-42e0-af76-f0f604a2d051 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:12.784305 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6888361-2546-41aa-b92c-545433c694a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:12.785644 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6888361-2546-41aa-b92c-545433c694a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:12.785968 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6888361-2546-41aa-b92c-545433c694a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:12.786247 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6888361-2546-41aa-b92c-545433c694a7 admin admin] PATH_INFO: `/v3/projects/491ea0fd07644de68f0110114edf9b26` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:12.787682 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6888361-2546-41aa-b92c-545433c694a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=491ea0fd07644de68f0110114edf9b26)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:12.789852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6888361-2546-41aa-b92c-545433c694a7 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:12.850671 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-e6888361-2546-41aa-b92c-545433c694a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '491ea0fd07644de68f0110114edf9b26', 'request_id': 'req-e6888361-2546-41aa-b92c-545433c694a7'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:12.859518 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-e6888361-2546-41aa-b92c-545433c694a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '491ea0fd07644de68f0110114edf9b26', 'request_id': 'req-e6888361-2546-41aa-b92c-545433c694a7'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:12.871116 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-e6888361-2546-41aa-b92c-545433c694a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '491ea0fd07644de68f0110114edf9b26', 'request_id': 'req-e6888361-2546-41aa-b92c-545433c694a7'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:12.883906 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-e6888361-2546-41aa-b92c-545433c694a7 admin admin] The token cache is being invalidate because project 491ea0fd07644de68f0110114edf9b26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:22:12.884325 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-e6888361-2546-41aa-b92c-545433c694a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:12.885141 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-e6888361-2546-41aa-b92c-545433c694a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:12.886528 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347ce2cf-565d-49c0-a3ff-8cbd0123a916 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:12.886869 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347ce2cf-565d-49c0-a3ff-8cbd0123a916 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:12.887150 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347ce2cf-565d-49c0-a3ff-8cbd0123a916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:12.887276 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1213/2436] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:22:12 2022] DELETE /identity/v3/projects/491ea0fd07644de68f0110114edf9b26 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:12.890979 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6888361-2546-41aa-b92c-545433c694a7 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:12.894069 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-e6888361-2546-41aa-b92c-545433c694a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:12.907870 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-347ce2cf-565d-49c0-a3ff-8cbd0123a916 None None] MFA Rules not processed for user `e4b46dc825704fbc9cc2505805f22f94`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:12.909126 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-347ce2cf-565d-49c0-a3ff-8cbd0123a916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:12.969523 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6888361-2546-41aa-b92c-545433c694a7 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:12.974280 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5220afae-c66a-4afb-9326-9e7acb77964d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:12.975174 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220afae-c66a-4afb-9326-9e7acb77964d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:12.975456 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220afae-c66a-4afb-9326-9e7acb77964d admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:12.975728 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220afae-c66a-4afb-9326-9e7acb77964d admin admin] PATH_INFO: `/v3/users/04c7bae8cb104db2b613d515f9454ca3` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:12.977223 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5220afae-c66a-4afb-9326-9e7acb77964d admin admin] RBAC: Authorizing `identity:delete_user(user_id=04c7bae8cb104db2b613d515f9454ca3)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:12.978103 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1224/2437] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:12.979185 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5220afae-c66a-4afb-9326-9e7acb77964d admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:13.050910 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-5220afae-c66a-4afb-9326-9e7acb77964d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04c7bae8cb104db2b613d515f9454ca3'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:13.059336 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-5220afae-c66a-4afb-9326-9e7acb77964d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04c7bae8cb104db2b613d515f9454ca3'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:13.060147 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-5220afae-c66a-4afb-9326-9e7acb77964d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04c7bae8cb104db2b613d515f9454ca3'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:13.061867 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-5220afae-c66a-4afb-9326-9e7acb77964d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04c7bae8cb104db2b613d515f9454ca3'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:13.070658 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-5220afae-c66a-4afb-9326-9e7acb77964d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04c7bae8cb104db2b613d515f9454ca3'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:13.085601 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:13.092841 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1214/2438] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:22:12 2022] DELETE /identity/v3/users/04c7bae8cb104db2b613d515f9454ca3 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:13.391312 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347ce2cf-565d-49c0-a3ff-8cbd0123a916 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:13.408033 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347ce2cf-565d-49c0-a3ff-8cbd0123a916 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:13.473691 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddac9e5e-7934-4899-8a4c-77d73340b225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:13.474620 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac9e5e-7934-4899-8a4c-77d73340b225 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:13.474957 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac9e5e-7934-4899-8a4c-77d73340b225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:13.475942 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac9e5e-7934-4899-8a4c-77d73340b225 admin admin] PATH_INFO: `/v3/projects/1f2f8b0b514f41ed9a204f2587f9f503` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:13.477144 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddac9e5e-7934-4899-8a4c-77d73340b225 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f2f8b0b514f41ed9a204f2587f9f503)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:13.479864 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddac9e5e-7934-4899-8a4c-77d73340b225 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:13.549119 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ddac9e5e-7934-4899-8a4c-77d73340b225 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f2f8b0b514f41ed9a204f2587f9f503', 'request_id': 'req-ddac9e5e-7934-4899-8a4c-77d73340b225'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:13.564223 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ddac9e5e-7934-4899-8a4c-77d73340b225 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f2f8b0b514f41ed9a204f2587f9f503', 'request_id': 'req-ddac9e5e-7934-4899-8a4c-77d73340b225'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:13.581140 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ddac9e5e-7934-4899-8a4c-77d73340b225 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f2f8b0b514f41ed9a204f2587f9f503', 'request_id': 'req-ddac9e5e-7934-4899-8a4c-77d73340b225'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:13.598727 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ddac9e5e-7934-4899-8a4c-77d73340b225 admin admin] The token cache is being invalidate because project 1f2f8b0b514f41ed9a204f2587f9f503 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:22:13.599316 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ddac9e5e-7934-4899-8a4c-77d73340b225 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:13.600402 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ddac9e5e-7934-4899-8a4c-77d73340b225 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:13.603610 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1225/2439] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:22:13 2022] DELETE /identity/v3/projects/1f2f8b0b514f41ed9a204f2587f9f503 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:13.608528 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725d86c-c076-4081-840f-d451a7d0213c None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:13.609213 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725d86c-c076-4081-840f-d451a7d0213c None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:13.609716 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b725d86c-c076-4081-840f-d451a7d0213c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:13.639948 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-b725d86c-c076-4081-840f-d451a7d0213c None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:13.641341 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-b725d86c-c076-4081-840f-d451a7d0213c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:13.721242 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1215/2440] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:22:13.735817 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f63916-d938-49fb-acbc-f40d1b69c4a7 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:13.736249 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f63916-d938-49fb-acbc-f40d1b69c4a7 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:13.736529 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f63916-d938-49fb-acbc-f40d1b69c4a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:13.756969 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-03f63916-d938-49fb-acbc-f40d1b69c4a7 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:13.758568 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-03f63916-d938-49fb-acbc-f40d1b69c4a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:13.768260 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1226/2441] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:13.773050 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b725d86c-c076-4081-840f-d451a7d0213c None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:13.784709 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b725d86c-c076-4081-840f-d451a7d0213c None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:13.791760 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5340fb-dd83-442e-bef0-4fe37d0cf459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:13.792227 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5340fb-dd83-442e-bef0-4fe37d0cf459 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:13.792510 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5340fb-dd83-442e-bef0-4fe37d0cf459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:13.792858 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5340fb-dd83-442e-bef0-4fe37d0cf459 admin admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:13.793723 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5340fb-dd83-442e-bef0-4fe37d0cf459 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:22:13.794071 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5340fb-dd83-442e-bef0-4fe37d0cf459 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:13.795796 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5340fb-dd83-442e-bef0-4fe37d0cf459 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:13.811115 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1216/2442] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:22:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:22:13.815707 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f63916-d938-49fb-acbc-f40d1b69c4a7 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:13.826266 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f63916-d938-49fb-acbc-f40d1b69c4a7 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:13.831430 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f252380-6aac-462a-b9d9-0421b33faede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:13.832117 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f252380-6aac-462a-b9d9-0421b33faede admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:13.832385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f252380-6aac-462a-b9d9-0421b33faede admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:13.833037 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f252380-6aac-462a-b9d9-0421b33faede admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:13.833750 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f252380-6aac-462a-b9d9-0421b33faede admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:13.835878 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f252380-6aac-462a-b9d9-0421b33faede admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:13.868088 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1227/2443] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:22:13 2022] POST /identity/v3/projects => generated 393 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:22:13.869640 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5340fb-dd83-442e-bef0-4fe37d0cf459 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:13.880503 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5340fb-dd83-442e-bef0-4fe37d0cf459 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:13.947660 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e0d0f-3db4-49a2-92ae-57ce28f6ebad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:13.949921 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e0d0f-3db4-49a2-92ae-57ce28f6ebad admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:13.949921 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e0d0f-3db4-49a2-92ae-57ce28f6ebad admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:13.949921 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e0d0f-3db4-49a2-92ae-57ce28f6ebad admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:13.950244 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201e0d0f-3db4-49a2-92ae-57ce28f6ebad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:13.952143 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201e0d0f-3db4-49a2-92ae-57ce28f6ebad admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:13.981361 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1217/2444] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:22:13 2022] POST /identity/v3/users => generated 297 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:22:13.982247 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f252380-6aac-462a-b9d9-0421b33faede admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:13.995345 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f252380-6aac-462a-b9d9-0421b33faede admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:14.000390 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5f2b51-f1b4-49f2-9d8d-274f867a6426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:14.001072 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f2b51-f1b4-49f2-9d8d-274f867a6426 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:14.001351 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f2b51-f1b4-49f2-9d8d-274f867a6426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:14.001758 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f2b51-f1b4-49f2-9d8d-274f867a6426 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:14.002499 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5f2b51-f1b4-49f2-9d8d-274f867a6426 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:14.004287 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5f2b51-f1b4-49f2-9d8d-274f867a6426 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:14.006160 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:14.022096 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1228/2445] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:22:14.029182 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e0d0f-3db4-49a2-92ae-57ce28f6ebad admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:14.040049 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e0d0f-3db4-49a2-92ae-57ce28f6ebad admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:14.044887 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e1ba1-66ff-4db7-87c7-de7b4c6918ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:14.045696 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e1ba1-66ff-4db7-87c7-de7b4c6918ad admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:14.046119 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e1ba1-66ff-4db7-87c7-de7b4c6918ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:14.046455 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e1ba1-66ff-4db7-87c7-de7b4c6918ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:14.047348 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e1ba1-66ff-4db7-87c7-de7b4c6918ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:14.049136 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e1ba1-66ff-4db7-87c7-de7b4c6918ad admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:14.063817 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1218/2446] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:22:14.068266 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5f2b51-f1b4-49f2-9d8d-274f867a6426 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:14.079749 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5f2b51-f1b4-49f2-9d8d-274f867a6426 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:14.085489 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0715f287-c11c-48ee-8cad-6ecdd6f2e57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:14.086639 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0715f287-c11c-48ee-8cad-6ecdd6f2e57d admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:14.087088 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0715f287-c11c-48ee-8cad-6ecdd6f2e57d admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:14.087426 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0715f287-c11c-48ee-8cad-6ecdd6f2e57d admin admin] PATH_INFO: `/v3/projects/bdf5c499322644e7b1a97188ba71f27f/users/6b3b0b148764485fa2816dbab7316beb/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:14.103996 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0715f287-c11c-48ee-8cad-6ecdd6f2e57d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdf5c499322644e7b1a97188ba71f27f, user_id=6b3b0b148764485fa2816dbab7316beb, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:14.107173 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0715f287-c11c-48ee-8cad-6ecdd6f2e57d admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:14.125295 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1229/2447] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:22:14 2022] PUT /identity/v3/projects/bdf5c499322644e7b1a97188ba71f27f/users/6b3b0b148764485fa2816dbab7316beb/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:22:14.137439 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd290849-bead-4ca4-8acc-aaf0f9f7ef2e None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:14.137628 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd290849-bead-4ca4-8acc-aaf0f9f7ef2e None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:14.137861 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd290849-bead-4ca4-8acc-aaf0f9f7ef2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:14.166502 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-cd290849-bead-4ca4-8acc-aaf0f9f7ef2e None None] MFA Rules not processed for user `6b3b0b148764485fa2816dbab7316beb`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:14.167867 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-cd290849-bead-4ca4-8acc-aaf0f9f7ef2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:14.261500 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:14.277105 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1219/2448] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:14.280054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0715f287-c11c-48ee-8cad-6ecdd6f2e57d admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:14.394217 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0715f287-c11c-48ee-8cad-6ecdd6f2e57d admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:14.401201 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e52f5-63d1-49c4-91df-48d2b553f52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:14.402187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e52f5-63d1-49c4-91df-48d2b553f52e admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:14.402669 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e52f5-63d1-49c4-91df-48d2b553f52e admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:14.403132 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26e52f5-63d1-49c4-91df-48d2b553f52e admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:14.404178 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26e52f5-63d1-49c4-91df-48d2b553f52e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:14.407168 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26e52f5-63d1-49c4-91df-48d2b553f52e admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:14.455123 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1230/2449] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:22:14 2022] POST /identity/v3/projects => generated 393 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:22:14.459739 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd290849-bead-4ca4-8acc-aaf0f9f7ef2e None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:14.482135 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd290849-bead-4ca4-8acc-aaf0f9f7ef2e None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:14.556835 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c9d8b-7e9d-4904-b50c-d9742a72a197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:14.557555 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c9d8b-7e9d-4904-b50c-d9742a72a197 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:14.557828 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c9d8b-7e9d-4904-b50c-d9742a72a197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:14.558119 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c9d8b-7e9d-4904-b50c-d9742a72a197 admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:14.559002 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c9d8b-7e9d-4904-b50c-d9742a72a197 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:14.561901 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c9d8b-7e9d-4904-b50c-d9742a72a197 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:14.591017 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1220/2450] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:22:14 2022] POST /identity/v3/users => generated 303 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:22:14.594770 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e52f5-63d1-49c4-91df-48d2b553f52e admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:14.608906 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26e52f5-63d1-49c4-91df-48d2b553f52e admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:14.616889 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22980da-5d62-4492-8a31-34b4342fd6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:14.617607 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22980da-5d62-4492-8a31-34b4342fd6d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:14.618661 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22980da-5d62-4492-8a31-34b4342fd6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:14.618970 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22980da-5d62-4492-8a31-34b4342fd6d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:14.619886 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22980da-5d62-4492-8a31-34b4342fd6d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:14.622392 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22980da-5d62-4492-8a31-34b4342fd6d0 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:14.641976 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c9d8b-7e9d-4904-b50c-d9742a72a197 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:14.649431 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1231/2451] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:22:14.654455 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c9d8b-7e9d-4904-b50c-d9742a72a197 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:14.660375 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defda311-17bd-4f92-ad31-334223054895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:14.661483 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defda311-17bd-4f92-ad31-334223054895 admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:14.661845 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defda311-17bd-4f92-ad31-334223054895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:14.662202 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defda311-17bd-4f92-ad31-334223054895 admin admin] PATH_INFO: `/v3/projects/7d2913158e7d4be3983089561326514f/users/983d231b0f054303a79b2b645d6cf5f5/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:14.682443 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defda311-17bd-4f92-ad31-334223054895 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d2913158e7d4be3983089561326514f, user_id=983d231b0f054303a79b2b645d6cf5f5, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:14.686249 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defda311-17bd-4f92-ad31-334223054895 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:14.706864 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1221/2452] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:22:14 2022] PUT /identity/v3/projects/7d2913158e7d4be3983089561326514f/users/983d231b0f054303a79b2b645d6cf5f5/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:14.711619 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22980da-5d62-4492-8a31-34b4342fd6d0 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:14.724231 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22980da-5d62-4492-8a31-34b4342fd6d0 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:14.797965 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44476f5-5c91-4c01-bdc4-038e84c68530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:14.798787 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44476f5-5c91-4c01-bdc4-038e84c68530 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:14.799106 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44476f5-5c91-4c01-bdc4-038e84c68530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:14.799456 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44476f5-5c91-4c01-bdc4-038e84c68530 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:14.800459 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44476f5-5c91-4c01-bdc4-038e84c68530 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:14.802492 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44476f5-5c91-4c01-bdc4-038e84c68530 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:14.819622 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1232/2453] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:22:14.824505 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defda311-17bd-4f92-ad31-334223054895 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:14.842312 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defda311-17bd-4f92-ad31-334223054895 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:14.851308 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ec084-ec28-4ff9-9359-7f79a60eec86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:14.853385 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ec084-ec28-4ff9-9359-7f79a60eec86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:14.853739 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ec084-ec28-4ff9-9359-7f79a60eec86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:14.854842 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618ec084-ec28-4ff9-9359-7f79a60eec86 admin admin] PATH_INFO: `/v3/projects/7d2913158e7d4be3983089561326514f/users/983d231b0f054303a79b2b645d6cf5f5/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:14.857908 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618ec084-ec28-4ff9-9359-7f79a60eec86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d2913158e7d4be3983089561326514f, user_id=983d231b0f054303a79b2b645d6cf5f5, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:14.861608 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618ec084-ec28-4ff9-9359-7f79a60eec86 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:14.875366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1222/2454] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:22:14 2022] PUT /identity/v3/projects/7d2913158e7d4be3983089561326514f/users/983d231b0f054303a79b2b645d6cf5f5/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:22:14.886362 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f1b89-7657-4d47-aa90-96013808439a None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:14.886362 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f1b89-7657-4d47-aa90-96013808439a None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:14.886793 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f1b89-7657-4d47-aa90-96013808439a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:14.922628 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-b26f1b89-7657-4d47-aa90-96013808439a None None] MFA Rules not processed for user `983d231b0f054303a79b2b645d6cf5f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:14.924274 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-b26f1b89-7657-4d47-aa90-96013808439a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:14.974896 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:15.035899 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1233/2455] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:14 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:15.049581 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ec084-ec28-4ff9-9359-7f79a60eec86 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:15.051389 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-618ec084-ec28-4ff9-9359-7f79a60eec86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:15.112822 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618ec084-ec28-4ff9-9359-7f79a60eec86 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:15.116617 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0e3f8-b7f7-4555-8144-45b462ae60f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:15.117405 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0e3f8-b7f7-4555-8144-45b462ae60f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:15.117689 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0e3f8-b7f7-4555-8144-45b462ae60f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:15.117964 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0e3f8-b7f7-4555-8144-45b462ae60f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:15.127971 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f0e3f8-b7f7-4555-8144-45b462ae60f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:15.129466 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f0e3f8-b7f7-4555-8144-45b462ae60f7 service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:15.140204 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1223/2456] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:15 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:22:15.190976 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f1b89-7657-4d47-aa90-96013808439a None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:15.193314 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-b26f1b89-7657-4d47-aa90-96013808439a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:15.212284 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f1b89-7657-4d47-aa90-96013808439a None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:15.216237 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e8eeea-313d-4917-8914-3ba56daf6403 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:15.216887 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8eeea-313d-4917-8914-3ba56daf6403 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:15.217183 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8eeea-313d-4917-8914-3ba56daf6403 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:15.217452 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8eeea-313d-4917-8914-3ba56daf6403 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:15.285735 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e8eeea-313d-4917-8914-3ba56daf6403 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:15.286845 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e8eeea-313d-4917-8914-3ba56daf6403 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:15.298022 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1234/2457] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:22:16.422969 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0e3f8-b7f7-4555-8144-45b462ae60f7 service cinder] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:16.425363 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-02f0e3f8-b7f7-4555-8144-45b462ae60f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:16.491792 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:16.520354 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0e3f8-b7f7-4555-8144-45b462ae60f7 service cinder] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:16.527500 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e020a8c3-e181-4ede-8a10-ea83ab6d5a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:16.528568 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020a8c3-e181-4ede-8a10-ea83ab6d5a1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:16.528962 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020a8c3-e181-4ede-8a10-ea83ab6d5a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:16.529335 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020a8c3-e181-4ede-8a10-ea83ab6d5a1c admin admin] PATH_INFO: `/v3/users/fff9f1fc84a54b83a7a384965f40c9d9` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:16.531070 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e020a8c3-e181-4ede-8a10-ea83ab6d5a1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=fff9f1fc84a54b83a7a384965f40c9d9)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:16.533129 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e020a8c3-e181-4ede-8a10-ea83ab6d5a1c admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:16.633419 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-e020a8c3-e181-4ede-8a10-ea83ab6d5a1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fff9f1fc84a54b83a7a384965f40c9d9'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:16.650321 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-e020a8c3-e181-4ede-8a10-ea83ab6d5a1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fff9f1fc84a54b83a7a384965f40c9d9'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:16.651144 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-e020a8c3-e181-4ede-8a10-ea83ab6d5a1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fff9f1fc84a54b83a7a384965f40c9d9'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:16.651751 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-e020a8c3-e181-4ede-8a10-ea83ab6d5a1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fff9f1fc84a54b83a7a384965f40c9d9'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:16.664344 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-e020a8c3-e181-4ede-8a10-ea83ab6d5a1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fff9f1fc84a54b83a7a384965f40c9d9'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:16.714913 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1224/2458] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:22:16 2022] DELETE /identity/v3/users/fff9f1fc84a54b83a7a384965f40c9d9 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:16.963003 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e8eeea-313d-4917-8914-3ba56daf6403 service cinder] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:16.990349 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e8eeea-313d-4917-8914-3ba56daf6403 service cinder] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:17.074750 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab66622-68b6-4c90-a131-033b5df351f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:17.076365 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab66622-68b6-4c90-a131-033b5df351f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:17.076365 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab66622-68b6-4c90-a131-033b5df351f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:17.076941 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab66622-68b6-4c90-a131-033b5df351f6 admin admin] PATH_INFO: `/v3/projects/4a74724882c84e8f893f5be0f59a9581` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:17.078245 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab66622-68b6-4c90-a131-033b5df351f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a74724882c84e8f893f5be0f59a9581)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:17.080348 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab66622-68b6-4c90-a131-033b5df351f6 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:17.165482 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-cab66622-68b6-4c90-a131-033b5df351f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a74724882c84e8f893f5be0f59a9581', 'request_id': 'req-cab66622-68b6-4c90-a131-033b5df351f6'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:17.174527 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-cab66622-68b6-4c90-a131-033b5df351f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a74724882c84e8f893f5be0f59a9581', 'request_id': 'req-cab66622-68b6-4c90-a131-033b5df351f6'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:17.189893 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-cab66622-68b6-4c90-a131-033b5df351f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a74724882c84e8f893f5be0f59a9581', 'request_id': 'req-cab66622-68b6-4c90-a131-033b5df351f6'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:17.199397 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:17.215813 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-cab66622-68b6-4c90-a131-033b5df351f6 admin admin] The token cache is being invalidate because project 4a74724882c84e8f893f5be0f59a9581 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:22:17.216972 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-cab66622-68b6-4c90-a131-033b5df351f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:17.217912 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-cab66622-68b6-4c90-a131-033b5df351f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:17.220649 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1235/2459] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:22:16 2022] DELETE /identity/v3/projects/4a74724882c84e8f893f5be0f59a9581 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:17.225276 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde5f0b-2ae7-4ae4-b7b3-07eb45237612 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:17.225637 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde5f0b-2ae7-4ae4-b7b3-07eb45237612 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:17.226035 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fde5f0b-2ae7-4ae4-b7b3-07eb45237612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:17.245858 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-4fde5f0b-2ae7-4ae4-b7b3-07eb45237612 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:17.248837 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-4fde5f0b-2ae7-4ae4-b7b3-07eb45237612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:17.342453 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1225/2460] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:17.351643 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d417948-dc06-47e0-a2ff-5b0aaaddb57a None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:17.352030 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d417948-dc06-47e0-a2ff-5b0aaaddb57a None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:17.352597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d417948-dc06-47e0-a2ff-5b0aaaddb57a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:17.384759 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-1d417948-dc06-47e0-a2ff-5b0aaaddb57a None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:17.386181 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-1d417948-dc06-47e0-a2ff-5b0aaaddb57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:17.396233 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1236/2461] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:17.401523 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fde5f0b-2ae7-4ae4-b7b3-07eb45237612 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:17.438017 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fde5f0b-2ae7-4ae4-b7b3-07eb45237612 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:17.451618 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d782d24b-5bab-4a70-90f6-96c6448590a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:17.452661 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782d24b-5bab-4a70-90f6-96c6448590a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:17.453111 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782d24b-5bab-4a70-90f6-96c6448590a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:17.453576 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782d24b-5bab-4a70-90f6-96c6448590a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:17.454749 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d782d24b-5bab-4a70-90f6-96c6448590a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:22:17.455119 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d782d24b-5bab-4a70-90f6-96c6448590a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:17.458281 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d782d24b-5bab-4a70-90f6-96c6448590a0 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:17.478721 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1226/2462] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:22:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:22:17.483339 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d417948-dc06-47e0-a2ff-5b0aaaddb57a None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:17.494662 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d417948-dc06-47e0-a2ff-5b0aaaddb57a None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:17.499611 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d8fa0-ae54-483c-b90d-07d49646e4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:17.500412 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d8fa0-ae54-483c-b90d-07d49646e4a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:17.500809 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d8fa0-ae54-483c-b90d-07d49646e4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:17.501209 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d8fa0-ae54-483c-b90d-07d49646e4a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:17.502027 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2d8fa0-ae54-483c-b90d-07d49646e4a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:17.504066 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2d8fa0-ae54-483c-b90d-07d49646e4a7 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:17.544356 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1237/2463] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:22:17 2022] POST /identity/v3/projects => generated 403 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:22:17.545621 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d782d24b-5bab-4a70-90f6-96c6448590a0 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:17.561840 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d782d24b-5bab-4a70-90f6-96c6448590a0 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:17.682958 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:17.702031 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7860d84f-3184-4c16-9253-501301b42a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:17.702723 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860d84f-3184-4c16-9253-501301b42a59 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:17.703227 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860d84f-3184-4c16-9253-501301b42a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:17.703323 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860d84f-3184-4c16-9253-501301b42a59 admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:17.704037 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7860d84f-3184-4c16-9253-501301b42a59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:17.705889 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7860d84f-3184-4c16-9253-501301b42a59 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:17.734288 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1227/2464] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:22:17 2022] POST /identity/v3/users => generated 302 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:22:17.738199 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d8fa0-ae54-483c-b90d-07d49646e4a7 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:17.753741 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d8fa0-ae54-483c-b90d-07d49646e4a7 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:17.759006 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dea366b-6ea9-492b-9f03-a3ce2ebfad5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:17.759788 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea366b-6ea9-492b-9f03-a3ce2ebfad5f admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:17.760139 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea366b-6ea9-492b-9f03-a3ce2ebfad5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:17.760471 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea366b-6ea9-492b-9f03-a3ce2ebfad5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:17.761321 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dea366b-6ea9-492b-9f03-a3ce2ebfad5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:17.763461 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dea366b-6ea9-492b-9f03-a3ce2ebfad5f admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:17.777296 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1238/2465] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:22:17.781502 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7860d84f-3184-4c16-9253-501301b42a59 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:17.794004 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7860d84f-3184-4c16-9253-501301b42a59 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:17.799358 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd5a146-a157-4063-87c1-fd8b8adf7826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:17.799674 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5a146-a157-4063-87c1-fd8b8adf7826 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:17.799964 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5a146-a157-4063-87c1-fd8b8adf7826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:17.800246 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5a146-a157-4063-87c1-fd8b8adf7826 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:17.801225 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd5a146-a157-4063-87c1-fd8b8adf7826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:17.802750 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd5a146-a157-4063-87c1-fd8b8adf7826 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:17.823003 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1228/2466] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:22:17.823852 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dea366b-6ea9-492b-9f03-a3ce2ebfad5f admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:17.835054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dea366b-6ea9-492b-9f03-a3ce2ebfad5f admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:17.841236 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c43eda-9185-43e4-bd04-c582716dbb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:17.842373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c43eda-9185-43e4-bd04-c582716dbb9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:17.842826 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c43eda-9185-43e4-bd04-c582716dbb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:17.843492 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c43eda-9185-43e4-bd04-c582716dbb9f admin admin] PATH_INFO: `/v3/projects/d06efe505ce0436ebdfdff455cbdc6a4/users/ecec201ef78b446ba55e019cee52a816/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:17.864851 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c43eda-9185-43e4-bd04-c582716dbb9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d06efe505ce0436ebdfdff455cbdc6a4, user_id=ecec201ef78b446ba55e019cee52a816, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:17.868864 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c43eda-9185-43e4-bd04-c582716dbb9f admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:17.889095 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1239/2467] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:22:17 2022] PUT /identity/v3/projects/d06efe505ce0436ebdfdff455cbdc6a4/users/ecec201ef78b446ba55e019cee52a816/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:17.901745 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba1e89-da9f-4dfd-bc86-a814d06e1f46 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:17.902320 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba1e89-da9f-4dfd-bc86-a814d06e1f46 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:17.902793 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba1e89-da9f-4dfd-bc86-a814d06e1f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:17.951855 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-41ba1e89-da9f-4dfd-bc86-a814d06e1f46 None None] MFA Rules not processed for user `ecec201ef78b446ba55e019cee52a816`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:17.954123 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-41ba1e89-da9f-4dfd-bc86-a814d06e1f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:18.035847 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1229/2468] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:17 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:22:18.040421 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c43eda-9185-43e4-bd04-c582716dbb9f admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:18.112400 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c43eda-9185-43e4-bd04-c582716dbb9f admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:18.118262 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e20e8f-8ff1-4d7c-97c9-1867146cb1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:18.118950 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e20e8f-8ff1-4d7c-97c9-1867146cb1e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:18.119265 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e20e8f-8ff1-4d7c-97c9-1867146cb1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:18.119533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e20e8f-8ff1-4d7c-97c9-1867146cb1e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:18.120260 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e20e8f-8ff1-4d7c-97c9-1867146cb1e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:18.122279 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e20e8f-8ff1-4d7c-97c9-1867146cb1e4 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:18.153730 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1240/2469] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:22:18 2022] POST /identity/v3/projects => generated 405 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:22:18.158054 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba1e89-da9f-4dfd-bc86-a814d06e1f46 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:18.170655 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba1e89-da9f-4dfd-bc86-a814d06e1f46 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:18.246760 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85eae58-e7f8-4b2a-b091-421b843c19fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:18.247475 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85eae58-e7f8-4b2a-b091-421b843c19fe admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:18.247757 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85eae58-e7f8-4b2a-b091-421b843c19fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:18.248039 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85eae58-e7f8-4b2a-b091-421b843c19fe admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:18.248761 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85eae58-e7f8-4b2a-b091-421b843c19fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:18.250825 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85eae58-e7f8-4b2a-b091-421b843c19fe admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:18.274626 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1230/2470] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:22:18 2022] POST /identity/v3/users => generated 309 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:22:18.278972 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e20e8f-8ff1-4d7c-97c9-1867146cb1e4 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:18.295215 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e20e8f-8ff1-4d7c-97c9-1867146cb1e4 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:18.304368 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d5d7c9-c066-4020-870d-f0d0a4f3b306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:18.305546 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5d7c9-c066-4020-870d-f0d0a4f3b306 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:18.305660 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5d7c9-c066-4020-870d-f0d0a4f3b306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:18.306310 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5d7c9-c066-4020-870d-f0d0a4f3b306 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:18.306668 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d5d7c9-c066-4020-870d-f0d0a4f3b306 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:18.308311 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d5d7c9-c066-4020-870d-f0d0a4f3b306 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:18.319761 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1241/2471] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:22:18.325777 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85eae58-e7f8-4b2a-b091-421b843c19fe admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:18.338263 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85eae58-e7f8-4b2a-b091-421b843c19fe admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:18.343847 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5c3b0-e4b1-4079-86a3-9349737374e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:18.344823 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5c3b0-e4b1-4079-86a3-9349737374e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:18.345198 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5c3b0-e4b1-4079-86a3-9349737374e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:18.345536 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5c3b0-e4b1-4079-86a3-9349737374e8 admin admin] PATH_INFO: `/v3/projects/da5a234cca614154886e2e2b19360dc6/users/9629052c17db4bcab21593408f5a8528/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:18.362202 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e5c3b0-e4b1-4079-86a3-9349737374e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da5a234cca614154886e2e2b19360dc6, user_id=9629052c17db4bcab21593408f5a8528, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:18.364886 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e5c3b0-e4b1-4079-86a3-9349737374e8 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:18.381635 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1231/2472] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:22:18 2022] PUT /identity/v3/projects/da5a234cca614154886e2e2b19360dc6/users/9629052c17db4bcab21593408f5a8528/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:18.386029 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d5d7c9-c066-4020-870d-f0d0a4f3b306 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:18.397762 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d5d7c9-c066-4020-870d-f0d0a4f3b306 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:18.461302 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87296aec-8b3f-47f5-b1e3-36adb2dba8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:18.461989 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87296aec-8b3f-47f5-b1e3-36adb2dba8a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:18.462269 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87296aec-8b3f-47f5-b1e3-36adb2dba8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:18.462542 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87296aec-8b3f-47f5-b1e3-36adb2dba8a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:18.463276 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87296aec-8b3f-47f5-b1e3-36adb2dba8a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:18.464947 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87296aec-8b3f-47f5-b1e3-36adb2dba8a3 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:18.475633 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1242/2473] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:22:18.480312 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5c3b0-e4b1-4079-86a3-9349737374e8 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:18.491071 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5c3b0-e4b1-4079-86a3-9349737374e8 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:18.495998 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a709b6-7ed6-4b91-b697-2fa149c58c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:18.496953 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a709b6-7ed6-4b91-b697-2fa149c58c6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:18.497399 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a709b6-7ed6-4b91-b697-2fa149c58c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:18.497689 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a709b6-7ed6-4b91-b697-2fa149c58c6e admin admin] PATH_INFO: `/v3/projects/da5a234cca614154886e2e2b19360dc6/users/9629052c17db4bcab21593408f5a8528/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:18.499826 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a709b6-7ed6-4b91-b697-2fa149c58c6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=da5a234cca614154886e2e2b19360dc6, user_id=9629052c17db4bcab21593408f5a8528, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:18.502546 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a709b6-7ed6-4b91-b697-2fa149c58c6e admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:18.514170 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1232/2474] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:22:18 2022] PUT /identity/v3/projects/da5a234cca614154886e2e2b19360dc6/users/9629052c17db4bcab21593408f5a8528/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:18.525898 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14eddfb-9d40-4e13-b1e6-93fc68e72437 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:18.526086 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14eddfb-9d40-4e13-b1e6-93fc68e72437 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:18.526363 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14eddfb-9d40-4e13-b1e6-93fc68e72437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:18.532905 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:18.559474 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-f14eddfb-9d40-4e13-b1e6-93fc68e72437 None None] MFA Rules not processed for user `9629052c17db4bcab21593408f5a8528`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:18.560682 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-f14eddfb-9d40-4e13-b1e6-93fc68e72437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:18.636747 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1243/2475] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:18 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:18.648109 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a709b6-7ed6-4b91-b697-2fa149c58c6e admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:18.649920 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-b8a709b6-7ed6-4b91-b697-2fa149c58c6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:18.669994 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:18.713667 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a709b6-7ed6-4b91-b697-2fa149c58c6e admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:18.717428 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b5f187-0fe9-423d-8de4-21f35dcb898c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:18.718090 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b5f187-0fe9-423d-8de4-21f35dcb898c service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:18.718368 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b5f187-0fe9-423d-8de4-21f35dcb898c service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:18.718640 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b5f187-0fe9-423d-8de4-21f35dcb898c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:18.728789 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b5f187-0fe9-423d-8de4-21f35dcb898c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:18.730282 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b5f187-0fe9-423d-8de4-21f35dcb898c service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:18.740960 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1233/2476] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:18 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:22:18.892449 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14eddfb-9d40-4e13-b1e6-93fc68e72437 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:18.959236 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14eddfb-9d40-4e13-b1e6-93fc68e72437 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:18.964354 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dd301c-9eae-41de-ad58-bc5f86f417fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:18.965265 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd301c-9eae-41de-ad58-bc5f86f417fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:18.965540 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd301c-9eae-41de-ad58-bc5f86f417fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:18.965818 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd301c-9eae-41de-ad58-bc5f86f417fa admin admin] PATH_INFO: `/v3/users/ecec201ef78b446ba55e019cee52a816` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:18.967077 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dd301c-9eae-41de-ad58-bc5f86f417fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecec201ef78b446ba55e019cee52a816)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:18.969046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dd301c-9eae-41de-ad58-bc5f86f417fa admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:19.021711 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-39dd301c-9eae-41de-ad58-bc5f86f417fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecec201ef78b446ba55e019cee52a816'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:19.029435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-39dd301c-9eae-41de-ad58-bc5f86f417fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecec201ef78b446ba55e019cee52a816'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:19.030271 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-39dd301c-9eae-41de-ad58-bc5f86f417fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecec201ef78b446ba55e019cee52a816'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:19.030925 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-39dd301c-9eae-41de-ad58-bc5f86f417fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecec201ef78b446ba55e019cee52a816'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:19.039547 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-39dd301c-9eae-41de-ad58-bc5f86f417fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecec201ef78b446ba55e019cee52a816'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:19.058536 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1244/2477] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:22:18 2022] DELETE /identity/v3/users/ecec201ef78b446ba55e019cee52a816 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:19.063316 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e4866d-6a54-4165-9e89-b01dc14c686e None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:19.063650 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e4866d-6a54-4165-9e89-b01dc14c686e None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:19.063989 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e4866d-6a54-4165-9e89-b01dc14c686e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:19.084226 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-19e4866d-6a54-4165-9e89-b01dc14c686e None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:19.086273 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-19e4866d-6a54-4165-9e89-b01dc14c686e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:19.159537 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1234/2478] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:19.169967 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dd301c-9eae-41de-ad58-bc5f86f417fa admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:19.171657 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-39dd301c-9eae-41de-ad58-bc5f86f417fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:19.182913 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:19.184770 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e4866d-6a54-4165-9e89-b01dc14c686e None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:19.186491 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-19e4866d-6a54-4165-9e89-b01dc14c686e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:19.200023 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e4866d-6a54-4165-9e89-b01dc14c686e None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:19.208392 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02c27fa-98bd-4ea5-9e68-78c52fe2759f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:19.209814 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c27fa-98bd-4ea5-9e68-78c52fe2759f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:19.210114 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c27fa-98bd-4ea5-9e68-78c52fe2759f admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:19.210463 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c27fa-98bd-4ea5-9e68-78c52fe2759f admin admin] PATH_INFO: `/v3/users/6b3b0b148764485fa2816dbab7316beb` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:19.212817 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02c27fa-98bd-4ea5-9e68-78c52fe2759f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b3b0b148764485fa2816dbab7316beb)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:19.214831 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02c27fa-98bd-4ea5-9e68-78c52fe2759f admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:19.248278 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dd301c-9eae-41de-ad58-bc5f86f417fa admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:19.253477 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b050594a-2770-4871-ad99-0b2011629439 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:19.254314 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b050594a-2770-4871-ad99-0b2011629439 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:19.254638 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b050594a-2770-4871-ad99-0b2011629439 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:19.254864 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b050594a-2770-4871-ad99-0b2011629439 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:19.270121 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b050594a-2770-4871-ad99-0b2011629439 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:19.272026 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b050594a-2770-4871-ad99-0b2011629439 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:19.273327 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-b02c27fa-98bd-4ea5-9e68-78c52fe2759f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b3b0b148764485fa2816dbab7316beb'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:19.281465 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-b02c27fa-98bd-4ea5-9e68-78c52fe2759f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b3b0b148764485fa2816dbab7316beb'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:19.282229 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-b02c27fa-98bd-4ea5-9e68-78c52fe2759f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b3b0b148764485fa2816dbab7316beb'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:19.282842 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-b02c27fa-98bd-4ea5-9e68-78c52fe2759f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b3b0b148764485fa2816dbab7316beb'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:19.283252 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1245/2479] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:22:19.291994 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-b02c27fa-98bd-4ea5-9e68-78c52fe2759f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b3b0b148764485fa2816dbab7316beb'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:19.311299 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1235/2480] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:22:19 2022] DELETE /identity/v3/users/6b3b0b148764485fa2816dbab7316beb => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:19.316148 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd69f6a-d598-4124-9303-1011401f2f0f None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:19.316440 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd69f6a-d598-4124-9303-1011401f2f0f None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:19.316718 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd69f6a-d598-4124-9303-1011401f2f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:19.336354 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-dcd69f6a-d598-4124-9303-1011401f2f0f None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:19.337611 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-dcd69f6a-d598-4124-9303-1011401f2f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:19.402538 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1246/2481] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:19.413123 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02c27fa-98bd-4ea5-9e68-78c52fe2759f admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:19.414614 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-b02c27fa-98bd-4ea5-9e68-78c52fe2759f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:19.462352 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:19.488773 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02c27fa-98bd-4ea5-9e68-78c52fe2759f admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:19.493154 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a744b99d-b689-495f-9077-a8aa2eecfa7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:19.493819 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a744b99d-b689-495f-9077-a8aa2eecfa7e service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:19.494102 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a744b99d-b689-495f-9077-a8aa2eecfa7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:19.494380 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a744b99d-b689-495f-9077-a8aa2eecfa7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:19.504258 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a744b99d-b689-495f-9077-a8aa2eecfa7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:19.505734 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a744b99d-b689-495f-9077-a8aa2eecfa7e service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:19.516683 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1236/2482] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:22:19.624973 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd69f6a-d598-4124-9303-1011401f2f0f None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:19.626902 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-dcd69f6a-d598-4124-9303-1011401f2f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:19.640810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd69f6a-d598-4124-9303-1011401f2f0f None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:19.647180 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d897c5-9a50-44f9-bc18-785511c03fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:19.648216 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d897c5-9a50-44f9-bc18-785511c03fbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:19.648500 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d897c5-9a50-44f9-bc18-785511c03fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:19.648774 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d897c5-9a50-44f9-bc18-785511c03fbb admin admin] PATH_INFO: `/v3/projects/d06efe505ce0436ebdfdff455cbdc6a4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:19.650204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d897c5-9a50-44f9-bc18-785511c03fbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d06efe505ce0436ebdfdff455cbdc6a4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:19.652168 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d897c5-9a50-44f9-bc18-785511c03fbb admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:19.710901 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a7d897c5-9a50-44f9-bc18-785511c03fbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd06efe505ce0436ebdfdff455cbdc6a4', 'request_id': 'req-a7d897c5-9a50-44f9-bc18-785511c03fbb'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:19.717899 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a7d897c5-9a50-44f9-bc18-785511c03fbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd06efe505ce0436ebdfdff455cbdc6a4', 'request_id': 'req-a7d897c5-9a50-44f9-bc18-785511c03fbb'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:19.726487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a7d897c5-9a50-44f9-bc18-785511c03fbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd06efe505ce0436ebdfdff455cbdc6a4', 'request_id': 'req-a7d897c5-9a50-44f9-bc18-785511c03fbb'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:19.745716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a7d897c5-9a50-44f9-bc18-785511c03fbb admin admin] The token cache is being invalidate because project d06efe505ce0436ebdfdff455cbdc6a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:22:19.746020 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a7d897c5-9a50-44f9-bc18-785511c03fbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:19.746721 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a7d897c5-9a50-44f9-bc18-785511c03fbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:19.748469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1247/2483] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:22:19 2022] DELETE /identity/v3/projects/d06efe505ce0436ebdfdff455cbdc6a4 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:19.752835 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a744b99d-b689-495f-9077-a8aa2eecfa7e service neutron] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:19.763545 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a744b99d-b689-495f-9077-a8aa2eecfa7e service neutron] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:19.823730 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dbd77b-4764-4ac5-a483-fb5f78121063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:19.824631 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbd77b-4764-4ac5-a483-fb5f78121063 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:19.824908 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbd77b-4764-4ac5-a483-fb5f78121063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:19.825211 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbd77b-4764-4ac5-a483-fb5f78121063 admin admin] PATH_INFO: `/v3/users/9629052c17db4bcab21593408f5a8528` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:19.826410 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dbd77b-4764-4ac5-a483-fb5f78121063 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9629052c17db4bcab21593408f5a8528)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:19.828343 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dbd77b-4764-4ac5-a483-fb5f78121063 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:19.869446 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d897c5-9a50-44f9-bc18-785511c03fbb admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:19.871038 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a7d897c5-9a50-44f9-bc18-785511c03fbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:19.881997 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-93dbd77b-4764-4ac5-a483-fb5f78121063 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9629052c17db4bcab21593408f5a8528'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:19.886711 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d897c5-9a50-44f9-bc18-785511c03fbb admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:19.891627 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-93dbd77b-4764-4ac5-a483-fb5f78121063 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9629052c17db4bcab21593408f5a8528'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:19.891789 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a480bb09-df01-43e4-bb5a-2ea240bb50dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:19.892422 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-93dbd77b-4764-4ac5-a483-fb5f78121063 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9629052c17db4bcab21593408f5a8528'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:19.892949 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a480bb09-df01-43e4-bb5a-2ea240bb50dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:19.893156 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-93dbd77b-4764-4ac5-a483-fb5f78121063 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9629052c17db4bcab21593408f5a8528'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:19.893388 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a480bb09-df01-43e4-bb5a-2ea240bb50dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:19.893835 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a480bb09-df01-43e4-bb5a-2ea240bb50dd admin admin] PATH_INFO: `/v3/projects/bdf5c499322644e7b1a97188ba71f27f` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:19.895149 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a480bb09-df01-43e4-bb5a-2ea240bb50dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdf5c499322644e7b1a97188ba71f27f)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:19.897370 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a480bb09-df01-43e4-bb5a-2ea240bb50dd admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:19.910105 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-93dbd77b-4764-4ac5-a483-fb5f78121063 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9629052c17db4bcab21593408f5a8528'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:19.933709 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1237/2484] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:22:19 2022] DELETE /identity/v3/users/9629052c17db4bcab21593408f5a8528 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:19.939773 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:19.960093 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a480bb09-df01-43e4-bb5a-2ea240bb50dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdf5c499322644e7b1a97188ba71f27f', 'request_id': 'req-a480bb09-df01-43e4-bb5a-2ea240bb50dd'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:19.967630 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a480bb09-df01-43e4-bb5a-2ea240bb50dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdf5c499322644e7b1a97188ba71f27f', 'request_id': 'req-a480bb09-df01-43e4-bb5a-2ea240bb50dd'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:19.977383 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a480bb09-df01-43e4-bb5a-2ea240bb50dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdf5c499322644e7b1a97188ba71f27f', 'request_id': 'req-a480bb09-df01-43e4-bb5a-2ea240bb50dd'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:19.990198 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a480bb09-df01-43e4-bb5a-2ea240bb50dd admin admin] The token cache is being invalidate because project bdf5c499322644e7b1a97188ba71f27f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:22:19.992985 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a480bb09-df01-43e4-bb5a-2ea240bb50dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:19.993751 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a480bb09-df01-43e4-bb5a-2ea240bb50dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:19.995768 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1248/2485] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:22:19 2022] DELETE /identity/v3/projects/bdf5c499322644e7b1a97188ba71f27f => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:22:20.000383 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dbd77b-4764-4ac5-a483-fb5f78121063 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:20.002056 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-93dbd77b-4764-4ac5-a483-fb5f78121063 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:20.068158 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:20.084388 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dbd77b-4764-4ac5-a483-fb5f78121063 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:20.152333 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadcd017-4ecc-4de2-8528-bae0e442e456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:20.153300 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadcd017-4ecc-4de2-8528-bae0e442e456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:20.153616 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadcd017-4ecc-4de2-8528-bae0e442e456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:20.153866 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadcd017-4ecc-4de2-8528-bae0e442e456 admin admin] PATH_INFO: `/v3/users/983d231b0f054303a79b2b645d6cf5f5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:20.155201 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadcd017-4ecc-4de2-8528-bae0e442e456 admin admin] RBAC: Authorizing `identity:delete_user(user_id=983d231b0f054303a79b2b645d6cf5f5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:20.157202 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadcd017-4ecc-4de2-8528-bae0e442e456 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:20.212271 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-aadcd017-4ecc-4de2-8528-bae0e442e456 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '983d231b0f054303a79b2b645d6cf5f5'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:20.220229 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-aadcd017-4ecc-4de2-8528-bae0e442e456 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '983d231b0f054303a79b2b645d6cf5f5'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:20.221067 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-aadcd017-4ecc-4de2-8528-bae0e442e456 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '983d231b0f054303a79b2b645d6cf5f5'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:20.221730 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-aadcd017-4ecc-4de2-8528-bae0e442e456 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '983d231b0f054303a79b2b645d6cf5f5'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:20.230342 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a480bb09-df01-43e4-bb5a-2ea240bb50dd admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:20.231807 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a480bb09-df01-43e4-bb5a-2ea240bb50dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:20.232695 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-aadcd017-4ecc-4de2-8528-bae0e442e456 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '983d231b0f054303a79b2b645d6cf5f5'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:20.246219 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a480bb09-df01-43e4-bb5a-2ea240bb50dd admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:20.252050 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d840c58-ae21-48d4-8bf4-f95846d3ee44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:20.253386 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d840c58-ae21-48d4-8bf4-f95846d3ee44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:20.253646 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d840c58-ae21-48d4-8bf4-f95846d3ee44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:20.253923 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d840c58-ae21-48d4-8bf4-f95846d3ee44 admin admin] PATH_INFO: `/v3/projects/da5a234cca614154886e2e2b19360dc6` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:20.255145 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d840c58-ae21-48d4-8bf4-f95846d3ee44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da5a234cca614154886e2e2b19360dc6)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:20.256745 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1238/2486] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:22:19 2022] DELETE /identity/v3/users/983d231b0f054303a79b2b645d6cf5f5 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:20.257338 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d840c58-ae21-48d4-8bf4-f95846d3ee44 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:20.320563 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-4d840c58-ae21-48d4-8bf4-f95846d3ee44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da5a234cca614154886e2e2b19360dc6', 'request_id': 'req-4d840c58-ae21-48d4-8bf4-f95846d3ee44'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:20.328564 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-4d840c58-ae21-48d4-8bf4-f95846d3ee44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da5a234cca614154886e2e2b19360dc6', 'request_id': 'req-4d840c58-ae21-48d4-8bf4-f95846d3ee44'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:20.339657 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-4d840c58-ae21-48d4-8bf4-f95846d3ee44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da5a234cca614154886e2e2b19360dc6', 'request_id': 'req-4d840c58-ae21-48d4-8bf4-f95846d3ee44'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:20.353114 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-4d840c58-ae21-48d4-8bf4-f95846d3ee44 admin admin] The token cache is being invalidate because project da5a234cca614154886e2e2b19360dc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:22:20.353397 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-4d840c58-ae21-48d4-8bf4-f95846d3ee44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:20.354103 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-4d840c58-ae21-48d4-8bf4-f95846d3ee44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:20.356005 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1249/2487] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:22:20 2022] DELETE /identity/v3/projects/da5a234cca614154886e2e2b19360dc6 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:22:20.361196 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f2067-3b49-4314-8a83-142038657c01 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:20.361482 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f2067-3b49-4314-8a83-142038657c01 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:20.361758 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f2067-3b49-4314-8a83-142038657c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:20.382760 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-e58f2067-3b49-4314-8a83-142038657c01 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:20.384519 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-e58f2067-3b49-4314-8a83-142038657c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:20.454596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1239/2488] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:22:20.462594 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81465bd9-411c-4b1a-ba8f-3babe51d4236 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:20.462983 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81465bd9-411c-4b1a-ba8f-3babe51d4236 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:20.463330 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81465bd9-411c-4b1a-ba8f-3babe51d4236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:20.482349 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-81465bd9-411c-4b1a-ba8f-3babe51d4236 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:20.483574 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-81465bd9-411c-4b1a-ba8f-3babe51d4236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:20.496041 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1250/2489] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:22:20.497343 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f2067-3b49-4314-8a83-142038657c01 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:20.507917 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f2067-3b49-4314-8a83-142038657c01 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:20.515980 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8634f10d-bcf2-44cf-b98f-37f8d3e97f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:20.516816 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8634f10d-bcf2-44cf-b98f-37f8d3e97f64 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:20.517193 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8634f10d-bcf2-44cf-b98f-37f8d3e97f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:20.517580 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8634f10d-bcf2-44cf-b98f-37f8d3e97f64 admin admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:20.518434 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8634f10d-bcf2-44cf-b98f-37f8d3e97f64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:22:20.518835 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8634f10d-bcf2-44cf-b98f-37f8d3e97f64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:20.520530 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8634f10d-bcf2-44cf-b98f-37f8d3e97f64 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:20.533061 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1240/2490] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:22:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:22:20.537649 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81465bd9-411c-4b1a-ba8f-3babe51d4236 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:20.547623 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81465bd9-411c-4b1a-ba8f-3babe51d4236 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:20.552865 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756eae1-a73d-467d-b3a9-3283ef5551b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:20.553593 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756eae1-a73d-467d-b3a9-3283ef5551b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:20.553876 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756eae1-a73d-467d-b3a9-3283ef5551b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:20.554189 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756eae1-a73d-467d-b3a9-3283ef5551b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:20.554896 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5756eae1-a73d-467d-b3a9-3283ef5551b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:20.556823 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5756eae1-a73d-467d-b3a9-3283ef5551b9 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:20.590521 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1251/2491] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:22:20 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:22:20.591693 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8634f10d-bcf2-44cf-b98f-37f8d3e97f64 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:20.602947 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8634f10d-bcf2-44cf-b98f-37f8d3e97f64 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:20.606960 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:20.646120 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756eae1-a73d-467d-b3a9-3283ef5551b9 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:20.648498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-5756eae1-a73d-467d-b3a9-3283ef5551b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:20.671209 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cf8ee3-7e35-4d5b-a41f-f374b4879a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:20.671905 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf8ee3-7e35-4d5b-a41f-f374b4879a3a admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:20.672185 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf8ee3-7e35-4d5b-a41f-f374b4879a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:20.672599 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf8ee3-7e35-4d5b-a41f-f374b4879a3a admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:20.673883 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cf8ee3-7e35-4d5b-a41f-f374b4879a3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:20.676498 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cf8ee3-7e35-4d5b-a41f-f374b4879a3a admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:20.700977 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1241/2492] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:22:20 2022] POST /identity/v3/users => generated 294 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:22:20.703739 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cf8ee3-7e35-4d5b-a41f-f374b4879a3a admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:20.714192 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cf8ee3-7e35-4d5b-a41f-f374b4879a3a admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:20.718819 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524a6e81-f8cd-49df-b797-78c83e5eefd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:20.719491 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a6e81-f8cd-49df-b797-78c83e5eefd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:20.719770 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a6e81-f8cd-49df-b797-78c83e5eefd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:20.720042 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a6e81-f8cd-49df-b797-78c83e5eefd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:20.720773 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524a6e81-f8cd-49df-b797-78c83e5eefd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:20.722394 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524a6e81-f8cd-49df-b797-78c83e5eefd8 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:20.726345 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756eae1-a73d-467d-b3a9-3283ef5551b9 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:20.731229 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9312de2-64c6-4f50-ab08-38b537d5c139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:20.732103 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9312de2-64c6-4f50-ab08-38b537d5c139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:20.732387 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9312de2-64c6-4f50-ab08-38b537d5c139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:20.732663 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9312de2-64c6-4f50-ab08-38b537d5c139 admin admin] PATH_INFO: `/v3/projects/7d2913158e7d4be3983089561326514f` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:20.733759 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1242/2493] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:22:20.733879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9312de2-64c6-4f50-ab08-38b537d5c139 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d2913158e7d4be3983089561326514f)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:20.735819 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9312de2-64c6-4f50-ab08-38b537d5c139 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:20.737875 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524a6e81-f8cd-49df-b797-78c83e5eefd8 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:20.748019 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524a6e81-f8cd-49df-b797-78c83e5eefd8 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:20.748963 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:20.753554 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0492c0-c58b-43dc-aad8-d798ff7a8b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:20.754232 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0492c0-c58b-43dc-aad8-d798ff7a8b59 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:20.754516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0492c0-c58b-43dc-aad8-d798ff7a8b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:20.754796 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0492c0-c58b-43dc-aad8-d798ff7a8b59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:20.755710 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0492c0-c58b-43dc-aad8-d798ff7a8b59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:20.757349 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0492c0-c58b-43dc-aad8-d798ff7a8b59 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:20.770447 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1243/2494] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:22:20.773835 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0492c0-c58b-43dc-aad8-d798ff7a8b59 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:20.783724 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0492c0-c58b-43dc-aad8-d798ff7a8b59 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:20.793221 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c9312de2-64c6-4f50-ab08-38b537d5c139 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d2913158e7d4be3983089561326514f', 'request_id': 'req-c9312de2-64c6-4f50-ab08-38b537d5c139'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:20.800742 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c9312de2-64c6-4f50-ab08-38b537d5c139 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d2913158e7d4be3983089561326514f', 'request_id': 'req-c9312de2-64c6-4f50-ab08-38b537d5c139'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:20.809616 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c9312de2-64c6-4f50-ab08-38b537d5c139 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d2913158e7d4be3983089561326514f', 'request_id': 'req-c9312de2-64c6-4f50-ab08-38b537d5c139'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:20.821139 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c9312de2-64c6-4f50-ab08-38b537d5c139 admin admin] The token cache is being invalidate because project 7d2913158e7d4be3983089561326514f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:22:20.821435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c9312de2-64c6-4f50-ab08-38b537d5c139 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:20.822097 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-c9312de2-64c6-4f50-ab08-38b537d5c139 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:20.823680 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1252/2495] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:22:20 2022] DELETE /identity/v3/projects/7d2913158e7d4be3983089561326514f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:22:20.828630 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97486fcc-88ff-4d32-bd30-f9045ece3940 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:20.829110 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97486fcc-88ff-4d32-bd30-f9045ece3940 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:20.829396 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97486fcc-88ff-4d32-bd30-f9045ece3940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:20.845256 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ca770-f4fe-4e93-8fb3-9b975ec6ba71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:20.846058 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ca770-f4fe-4e93-8fb3-9b975ec6ba71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:20.846342 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ca770-f4fe-4e93-8fb3-9b975ec6ba71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:20.846614 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ca770-f4fe-4e93-8fb3-9b975ec6ba71 admin admin] PATH_INFO: `/v3/projects/a31610abf8444a3fb4b0cc98489915be/users/000ca70757c943a58329416fe4bcd94a/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:20.849123 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-97486fcc-88ff-4d32-bd30-f9045ece3940 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:20.850717 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-97486fcc-88ff-4d32-bd30-f9045ece3940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:20.860162 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1253/2496] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:22:20.864388 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ca770-f4fe-4e93-8fb3-9b975ec6ba71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a31610abf8444a3fb4b0cc98489915be, user_id=000ca70757c943a58329416fe4bcd94a, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:20.867650 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ca770-f4fe-4e93-8fb3-9b975ec6ba71 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:20.868191 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de03af9-cbda-49ef-af2a-e5bfaf2ca056 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:20.868483 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de03af9-cbda-49ef-af2a-e5bfaf2ca056 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:20.868764 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de03af9-cbda-49ef-af2a-e5bfaf2ca056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:20.882324 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1244/2497] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:22:20 2022] PUT /identity/v3/projects/a31610abf8444a3fb4b0cc98489915be/users/000ca70757c943a58329416fe4bcd94a/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:22:20.887775 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-6de03af9-cbda-49ef-af2a-e5bfaf2ca056 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:20.889014 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-6de03af9-cbda-49ef-af2a-e5bfaf2ca056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:20.894484 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701b32b-a374-411d-894f-283ec836b031 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:20.894756 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701b32b-a374-411d-894f-283ec836b031 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:20.895036 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701b32b-a374-411d-894f-283ec836b031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:20.923353 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-3701b32b-a374-411d-894f-283ec836b031 None None] MFA Rules not processed for user `000ca70757c943a58329416fe4bcd94a`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:20.924649 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-3701b32b-a374-411d-894f-283ec836b031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:20.953114 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1254/2498] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:22:20.957796 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de03af9-cbda-49ef-af2a-e5bfaf2ca056 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:20.967304 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de03af9-cbda-49ef-af2a-e5bfaf2ca056 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:20.972310 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:20.973817 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32334b57-295a-407b-bb80-7bc42d783105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:20.974872 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32334b57-295a-407b-bb80-7bc42d783105 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:20.975285 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32334b57-295a-407b-bb80-7bc42d783105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:20.975790 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32334b57-295a-407b-bb80-7bc42d783105 admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:20.976856 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32334b57-295a-407b-bb80-7bc42d783105 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:22:20.977383 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32334b57-295a-407b-bb80-7bc42d783105 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:20.980009 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32334b57-295a-407b-bb80-7bc42d783105 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:21.003868 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1255/2499] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:22:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:22:21.004083 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1245/2500] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:22:21.005059 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32334b57-295a-407b-bb80-7bc42d783105 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:21.006981 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701b32b-a374-411d-894f-283ec836b031 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:21.008918 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-3701b32b-a374-411d-894f-283ec836b031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:21.016696 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32334b57-295a-407b-bb80-7bc42d783105 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:21.022350 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c6e2b-f8c2-4ea3-be25-9f155baf0f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:21.022503 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701b32b-a374-411d-894f-283ec836b031 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:21.023375 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c6e2b-f8c2-4ea3-be25-9f155baf0f01 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:21.024084 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c6e2b-f8c2-4ea3-be25-9f155baf0f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:21.024367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c6e2b-f8c2-4ea3-be25-9f155baf0f01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:21.025135 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633c6e2b-f8c2-4ea3-be25-9f155baf0f01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:21.027027 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633c6e2b-f8c2-4ea3-be25-9f155baf0f01 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:21.032876 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40682aea-efb9-49e7-9242-ffcbb446a1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:21.033595 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40682aea-efb9-49e7-9242-ffcbb446a1e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:21.033876 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40682aea-efb9-49e7-9242-ffcbb446a1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:21.034152 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40682aea-efb9-49e7-9242-ffcbb446a1e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:21.034887 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40682aea-efb9-49e7-9242-ffcbb446a1e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:21.036900 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40682aea-efb9-49e7-9242-ffcbb446a1e9 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:21.057412 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1256/2501] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:22:21 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:22:21.062116 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c6e2b-f8c2-4ea3-be25-9f155baf0f01 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:21.065934 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1246/2502] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:22:21 2022] POST /identity/v3/projects => generated 387 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:22:21.069647 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40682aea-efb9-49e7-9242-ffcbb446a1e9 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:21.135999 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c6e2b-f8c2-4ea3-be25-9f155baf0f01 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:21.141156 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8a8c29-ecbb-4445-9ea5-59f4764abe31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:21.141712 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a8c29-ecbb-4445-9ea5-59f4764abe31 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:21.142003 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a8c29-ecbb-4445-9ea5-59f4764abe31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:21.142282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8a8c29-ecbb-4445-9ea5-59f4764abe31 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:21.143052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8a8c29-ecbb-4445-9ea5-59f4764abe31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:21.143162 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40682aea-efb9-49e7-9242-ffcbb446a1e9 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:21.145017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8a8c29-ecbb-4445-9ea5-59f4764abe31 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:21.148376 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7388fcf-db86-47fa-87b0-5997b01bec75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:21.149012 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7388fcf-db86-47fa-87b0-5997b01bec75 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:21.149293 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7388fcf-db86-47fa-87b0-5997b01bec75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:21.149568 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7388fcf-db86-47fa-87b0-5997b01bec75 admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:21.150282 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7388fcf-db86-47fa-87b0-5997b01bec75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:21.152207 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:21.152368 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7388fcf-db86-47fa-87b0-5997b01bec75 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:21.168686 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1257/2503] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:22:21 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:22:21.173016 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8a8c29-ecbb-4445-9ea5-59f4764abe31 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:21.181003 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1247/2504] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:22:21 2022] POST /identity/v3/users => generated 294 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:22:21.182483 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7388fcf-db86-47fa-87b0-5997b01bec75 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:21.185461 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8a8c29-ecbb-4445-9ea5-59f4764abe31 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:21.193445 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7388fcf-db86-47fa-87b0-5997b01bec75 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:21.193835 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa49877-a9ce-4efe-b81f-74086de43393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:21.194531 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa49877-a9ce-4efe-b81f-74086de43393 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:21.195030 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa49877-a9ce-4efe-b81f-74086de43393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:21.195819 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa49877-a9ce-4efe-b81f-74086de43393 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:21.196591 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa49877-a9ce-4efe-b81f-74086de43393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:21.198270 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa49877-a9ce-4efe-b81f-74086de43393 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:21.199130 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c03ddc-aa6e-4366-81c9-b91181d21c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:21.200165 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c03ddc-aa6e-4366-81c9-b91181d21c76 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:21.200505 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c03ddc-aa6e-4366-81c9-b91181d21c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:21.201331 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c03ddc-aa6e-4366-81c9-b91181d21c76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:21.202235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c03ddc-aa6e-4366-81c9-b91181d21c76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:21.203984 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c03ddc-aa6e-4366-81c9-b91181d21c76 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:21.214273 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1258/2505] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:22:21.218483 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa49877-a9ce-4efe-b81f-74086de43393 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:21.220951 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1248/2506] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:22:21.225069 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c03ddc-aa6e-4366-81c9-b91181d21c76 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:21.229734 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa49877-a9ce-4efe-b81f-74086de43393 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:21.234507 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5557d8bf-b023-4444-9b96-f1bd1b381aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:21.235173 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557d8bf-b023-4444-9b96-f1bd1b381aa4 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:21.235457 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557d8bf-b023-4444-9b96-f1bd1b381aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:21.235736 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557d8bf-b023-4444-9b96-f1bd1b381aa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:21.236035 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c03ddc-aa6e-4366-81c9-b91181d21c76 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:21.236514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5557d8bf-b023-4444-9b96-f1bd1b381aa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:21.238179 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5557d8bf-b023-4444-9b96-f1bd1b381aa4 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:21.241243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9827e9-389c-41e3-b4eb-81599104c24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:21.242367 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9827e9-389c-41e3-b4eb-81599104c24a admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:21.242928 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9827e9-389c-41e3-b4eb-81599104c24a admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:21.243224 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9827e9-389c-41e3-b4eb-81599104c24a admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:21.244025 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9827e9-389c-41e3-b4eb-81599104c24a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:21.245727 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9827e9-389c-41e3-b4eb-81599104c24a admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:21.250725 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1259/2507] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:22:21.255043 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5557d8bf-b023-4444-9b96-f1bd1b381aa4 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:21.261239 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1249/2508] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:22:21.265446 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9827e9-389c-41e3-b4eb-81599104c24a admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:21.265683 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5557d8bf-b023-4444-9b96-f1bd1b381aa4 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:21.270570 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817f4f6b-fe5d-4d3a-a3a3-402f941471d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:21.271452 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f4f6b-fe5d-4d3a-a3a3-402f941471d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:21.272523 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f4f6b-fe5d-4d3a-a3a3-402f941471d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:21.272523 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f4f6b-fe5d-4d3a-a3a3-402f941471d0 admin admin] PATH_INFO: `/v3/projects/c53f85f9a90c4751b5f5b5412dddc507/users/93eb53f596674fa387c1ce83f898d05a/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:21.276067 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9827e9-389c-41e3-b4eb-81599104c24a admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:21.280817 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b815dd1e-a1b5-4c8a-9e84-22afdf4eeef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:21.281750 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815dd1e-a1b5-4c8a-9e84-22afdf4eeef0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:21.282024 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815dd1e-a1b5-4c8a-9e84-22afdf4eeef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:21.282297 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815dd1e-a1b5-4c8a-9e84-22afdf4eeef0 admin admin] PATH_INFO: `/v3/projects/fd50724cd4e64e5f826bd8afc44a4808/users/bd52c01f43794b7080fe736d6db70260/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:21.288733 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817f4f6b-fe5d-4d3a-a3a3-402f941471d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c53f85f9a90c4751b5f5b5412dddc507, user_id=93eb53f596674fa387c1ce83f898d05a, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:21.291479 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817f4f6b-fe5d-4d3a-a3a3-402f941471d0 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:21.299063 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b815dd1e-a1b5-4c8a-9e84-22afdf4eeef0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd50724cd4e64e5f826bd8afc44a4808, user_id=bd52c01f43794b7080fe736d6db70260, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:21.301750 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b815dd1e-a1b5-4c8a-9e84-22afdf4eeef0 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:21.306286 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1260/2509] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:22:21 2022] PUT /identity/v3/projects/c53f85f9a90c4751b5f5b5412dddc507/users/93eb53f596674fa387c1ce83f898d05a/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:21.316623 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1250/2510] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:22:21 2022] PUT /identity/v3/projects/fd50724cd4e64e5f826bd8afc44a4808/users/bd52c01f43794b7080fe736d6db70260/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:22:21.317632 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df73262-7f66-4e53-bb6a-232c32fb87b4 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:21.318265 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df73262-7f66-4e53-bb6a-232c32fb87b4 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:21.318265 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df73262-7f66-4e53-bb6a-232c32fb87b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:21.328271 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86bf38a-bf86-4e69-88a8-505fc7b871bf None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:21.328551 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86bf38a-bf86-4e69-88a8-505fc7b871bf None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:21.328824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86bf38a-bf86-4e69-88a8-505fc7b871bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:21.347305 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-2df73262-7f66-4e53-bb6a-232c32fb87b4 None None] MFA Rules not processed for user `93eb53f596674fa387c1ce83f898d05a`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:21.348551 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-2df73262-7f66-4e53-bb6a-232c32fb87b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:21.357858 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-a86bf38a-bf86-4e69-88a8-505fc7b871bf None None] MFA Rules not processed for user `bd52c01f43794b7080fe736d6db70260`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:21.359049 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-a86bf38a-bf86-4e69-88a8-505fc7b871bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:21.438900 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1261/2511] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:22:21.439196 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:21.440301 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df73262-7f66-4e53-bb6a-232c32fb87b4 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:21.448701 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1251/2512] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:22:21.453343 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86bf38a-bf86-4e69-88a8-505fc7b871bf None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:21.519443 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df73262-7f66-4e53-bb6a-232c32fb87b4 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:21.524434 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2807b-3f80-4c08-87b2-1f1b3adf445c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:21.525156 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2807b-3f80-4c08-87b2-1f1b3adf445c admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:21.525440 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2807b-3f80-4c08-87b2-1f1b3adf445c admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:21.525719 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2807b-3f80-4c08-87b2-1f1b3adf445c admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:21.526423 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa2807b-3f80-4c08-87b2-1f1b3adf445c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:21.527166 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86bf38a-bf86-4e69-88a8-505fc7b871bf None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:21.528363 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa2807b-3f80-4c08-87b2-1f1b3adf445c admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:21.533879 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4264a3dc-577b-49e4-8039-7a4947323b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:21.534637 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264a3dc-577b-49e4-8039-7a4947323b63 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:21.535091 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264a3dc-577b-49e4-8039-7a4947323b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:21.535431 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264a3dc-577b-49e4-8039-7a4947323b63 admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:21.536227 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4264a3dc-577b-49e4-8039-7a4947323b63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:21.538222 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4264a3dc-577b-49e4-8039-7a4947323b63 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:21.568997 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1262/2513] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:22:21 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:22:21.573771 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2807b-3f80-4c08-87b2-1f1b3adf445c admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:21.577223 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1252/2514] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:22:21 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:22:21.585218 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4264a3dc-577b-49e4-8039-7a4947323b63 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:21.600067 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:21.667690 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2807b-3f80-4c08-87b2-1f1b3adf445c admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:21.667961 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4264a3dc-577b-49e4-8039-7a4947323b63 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:21.672603 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189c4da5-3068-43ff-97af-d7bca962a4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:21.673314 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c4da5-3068-43ff-97af-d7bca962a4ce admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:21.674127 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c4da5-3068-43ff-97af-d7bca962a4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:21.674269 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc53d26e-e843-4a27-a26c-8965109186f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:21.674377 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c4da5-3068-43ff-97af-d7bca962a4ce admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:21.674582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53d26e-e843-4a27-a26c-8965109186f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:21.674704 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189c4da5-3068-43ff-97af-d7bca962a4ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:21.674870 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53d26e-e843-4a27-a26c-8965109186f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:21.675144 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53d26e-e843-4a27-a26c-8965109186f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:21.675994 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc53d26e-e843-4a27-a26c-8965109186f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:21.676626 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189c4da5-3068-43ff-97af-d7bca962a4ce admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:21.677979 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc53d26e-e843-4a27-a26c-8965109186f1 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:21.711925 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1263/2515] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:22:21 2022] POST /identity/v3/users => generated 299 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:22:21.719078 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189c4da5-3068-43ff-97af-d7bca962a4ce admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:21.727074 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1253/2516] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:22:21 2022] POST /identity/v3/users => generated 299 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:22:21.728523 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc53d26e-e843-4a27-a26c-8965109186f1 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:21.733233 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189c4da5-3068-43ff-97af-d7bca962a4ce admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:21.738342 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c570a93a-8d2f-451a-b7f7-5be70eb6d4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:21.738967 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc53d26e-e843-4a27-a26c-8965109186f1 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:21.739105 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570a93a-8d2f-451a-b7f7-5be70eb6d4ae admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:21.739288 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570a93a-8d2f-451a-b7f7-5be70eb6d4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:21.739564 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570a93a-8d2f-451a-b7f7-5be70eb6d4ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:21.740338 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c570a93a-8d2f-451a-b7f7-5be70eb6d4ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:21.742130 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c570a93a-8d2f-451a-b7f7-5be70eb6d4ae admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:21.746377 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3523b5-a222-41de-ba3a-71a8c7b26fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:21.746616 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3523b5-a222-41de-ba3a-71a8c7b26fbe admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:21.746892 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3523b5-a222-41de-ba3a-71a8c7b26fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:21.747160 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3523b5-a222-41de-ba3a-71a8c7b26fbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:21.748127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3523b5-a222-41de-ba3a-71a8c7b26fbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:21.749900 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3523b5-a222-41de-ba3a-71a8c7b26fbe admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:21.753439 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1264/2517] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:22:21.761563 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1254/2518] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:22:21.762253 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c570a93a-8d2f-451a-b7f7-5be70eb6d4ae admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:21.766595 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3523b5-a222-41de-ba3a-71a8c7b26fbe admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:21.775508 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c570a93a-8d2f-451a-b7f7-5be70eb6d4ae admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:21.778897 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3523b5-a222-41de-ba3a-71a8c7b26fbe admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:21.782514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b927baf-2e46-4437-adcc-ea9c357e2f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:21.783444 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b927baf-2e46-4437-adcc-ea9c357e2f85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:21.783604 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be7c65-52c9-40ca-8652-75bc8fe4eb40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:21.783712 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b927baf-2e46-4437-adcc-ea9c357e2f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:21.784002 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b927baf-2e46-4437-adcc-ea9c357e2f85 admin admin] PATH_INFO: `/v3/projects/8ec80a93546e4ca3a9222562c0423da6/users/a83cc7cd946946bbb355c217e6256d9e/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:21.784439 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be7c65-52c9-40ca-8652-75bc8fe4eb40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:21.784727 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be7c65-52c9-40ca-8652-75bc8fe4eb40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:21.785023 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be7c65-52c9-40ca-8652-75bc8fe4eb40 admin admin] PATH_INFO: `/v3/projects/a23d6eda15a7404d9259bb0e176d81a7/users/2125b3117ba04f88a891beace8ba8c23/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:21.800635 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b927baf-2e46-4437-adcc-ea9c357e2f85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ec80a93546e4ca3a9222562c0423da6, user_id=a83cc7cd946946bbb355c217e6256d9e, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:21.802648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5be7c65-52c9-40ca-8652-75bc8fe4eb40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a23d6eda15a7404d9259bb0e176d81a7, user_id=2125b3117ba04f88a891beace8ba8c23, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:21.803361 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b927baf-2e46-4437-adcc-ea9c357e2f85 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:21.807231 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5be7c65-52c9-40ca-8652-75bc8fe4eb40 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:21.822179 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1265/2519] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:22:21 2022] PUT /identity/v3/projects/8ec80a93546e4ca3a9222562c0423da6/users/a83cc7cd946946bbb355c217e6256d9e/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:21.826303 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b927baf-2e46-4437-adcc-ea9c357e2f85 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:21.827461 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1255/2520] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:22:21 2022] PUT /identity/v3/projects/a23d6eda15a7404d9259bb0e176d81a7/users/2125b3117ba04f88a891beace8ba8c23/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:22:21.831214 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be7c65-52c9-40ca-8652-75bc8fe4eb40 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:21.880260 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:21.895504 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b927baf-2e46-4437-adcc-ea9c357e2f85 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:21.900959 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95e1668-8080-44a2-9318-23f9e2230b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:21.901640 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e1668-8080-44a2-9318-23f9e2230b22 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:21.901929 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e1668-8080-44a2-9318-23f9e2230b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:21.902206 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95e1668-8080-44a2-9318-23f9e2230b22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:21.902968 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95e1668-8080-44a2-9318-23f9e2230b22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:21.904439 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be7c65-52c9-40ca-8652-75bc8fe4eb40 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:21.904576 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95e1668-8080-44a2-9318-23f9e2230b22 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:21.909752 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d208a8-5339-4c45-8767-d06efe1b6d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:21.910460 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d208a8-5339-4c45-8767-d06efe1b6d33 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:21.910759 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d208a8-5339-4c45-8767-d06efe1b6d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:21.911029 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d208a8-5339-4c45-8767-d06efe1b6d33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:21.911761 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d208a8-5339-4c45-8767-d06efe1b6d33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:21.913415 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d208a8-5339-4c45-8767-d06efe1b6d33 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:21.916332 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1266/2521] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:22:21.920696 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95e1668-8080-44a2-9318-23f9e2230b22 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:21.927983 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1256/2522] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:22:21.929082 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d208a8-5339-4c45-8767-d06efe1b6d33 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:21.932329 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95e1668-8080-44a2-9318-23f9e2230b22 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:21.938355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff7842a-a990-4c15-9035-ff5939ec7828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:21.939257 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7842a-a990-4c15-9035-ff5939ec7828 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:21.939532 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7842a-a990-4c15-9035-ff5939ec7828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:21.939797 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff7842a-a990-4c15-9035-ff5939ec7828 admin admin] PATH_INFO: `/v3/projects/8ec80a93546e4ca3a9222562c0423da6/users/a83cc7cd946946bbb355c217e6256d9e/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:21.940346 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d208a8-5339-4c45-8767-d06efe1b6d33 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:21.941827 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff7842a-a990-4c15-9035-ff5939ec7828 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ec80a93546e4ca3a9222562c0423da6, user_id=a83cc7cd946946bbb355c217e6256d9e, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:21.944481 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff7842a-a990-4c15-9035-ff5939ec7828 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:21.945195 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30753d65-e1ab-419f-bd30-d81fd318e603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:21.946182 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30753d65-e1ab-419f-bd30-d81fd318e603 admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:21.946459 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30753d65-e1ab-419f-bd30-d81fd318e603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:21.946738 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30753d65-e1ab-419f-bd30-d81fd318e603 admin admin] PATH_INFO: `/v3/projects/a23d6eda15a7404d9259bb0e176d81a7/users/2125b3117ba04f88a891beace8ba8c23/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:21.948739 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30753d65-e1ab-419f-bd30-d81fd318e603 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a23d6eda15a7404d9259bb0e176d81a7, user_id=2125b3117ba04f88a891beace8ba8c23, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:21.951465 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30753d65-e1ab-419f-bd30-d81fd318e603 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:21.955939 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1267/2523] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:22:21 2022] PUT /identity/v3/projects/8ec80a93546e4ca3a9222562c0423da6/users/a83cc7cd946946bbb355c217e6256d9e/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:22:21.966752 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b8ed2-b0fd-42db-a2c3-69d80034a740 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:21.967061 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1257/2524] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:22:21 2022] PUT /identity/v3/projects/a23d6eda15a7404d9259bb0e176d81a7/users/2125b3117ba04f88a891beace8ba8c23/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:21.967180 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b8ed2-b0fd-42db-a2c3-69d80034a740 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:21.967362 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b8ed2-b0fd-42db-a2c3-69d80034a740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:21.977402 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc25d6-80df-46a8-81cc-fe73eabe795d None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:21.977665 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc25d6-80df-46a8-81cc-fe73eabe795d None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:21.977919 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc25d6-80df-46a8-81cc-fe73eabe795d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:21.995694 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-fe7b8ed2-b0fd-42db-a2c3-69d80034a740 None None] MFA Rules not processed for user `a83cc7cd946946bbb355c217e6256d9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:21.996950 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-fe7b8ed2-b0fd-42db-a2c3-69d80034a740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:22.007870 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-0ddc25d6-80df-46a8-81cc-fe73eabe795d None None] MFA Rules not processed for user `2125b3117ba04f88a891beace8ba8c23`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:22.009172 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-0ddc25d6-80df-46a8-81cc-fe73eabe795d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:22.049272 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:22.076049 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1268/2525] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:21 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:22:22.081206 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1258/2526] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:21 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:22.087550 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b8ed2-b0fd-42db-a2c3-69d80034a740 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:22.089133 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-fe7b8ed2-b0fd-42db-a2c3-69d80034a740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:22.093019 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc25d6-80df-46a8-81cc-fe73eabe795d None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:22.094636 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-0ddc25d6-80df-46a8-81cc-fe73eabe795d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:22.151485 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b8ed2-b0fd-42db-a2c3-69d80034a740 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:22.154577 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc25d6-80df-46a8-81cc-fe73eabe795d None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:22.156908 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2465ebc-6cc7-462e-ac5c-ecaf591478a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:22.157691 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2465ebc-6cc7-462e-ac5c-ecaf591478a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:22.157999 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2465ebc-6cc7-462e-ac5c-ecaf591478a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:22.158261 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2465ebc-6cc7-462e-ac5c-ecaf591478a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:22.159950 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879ead94-e309-4f4a-8232-88efbd344a84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:22.161297 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ead94-e309-4f4a-8232-88efbd344a84 service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:22.162135 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ead94-e309-4f4a-8232-88efbd344a84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:22.162454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ead94-e309-4f4a-8232-88efbd344a84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:22.173985 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879ead94-e309-4f4a-8232-88efbd344a84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:22.175432 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879ead94-e309-4f4a-8232-88efbd344a84 service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:22.185680 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1259/2527] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:22 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:22:22.222549 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2465ebc-6cc7-462e-ac5c-ecaf591478a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:22.224015 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2465ebc-6cc7-462e-ac5c-ecaf591478a3 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:22.234196 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1269/2528] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:22:22.758781 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879ead94-e309-4f4a-8232-88efbd344a84 service cinder] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:22.768229 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879ead94-e309-4f4a-8232-88efbd344a84 service cinder] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:22.772249 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efad4fd8-e587-41e8-8d0c-48b4b1f8ca57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:22.772991 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad4fd8-e587-41e8-8d0c-48b4b1f8ca57 service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:22.773590 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad4fd8-e587-41e8-8d0c-48b4b1f8ca57 service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:22.774346 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad4fd8-e587-41e8-8d0c-48b4b1f8ca57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:22.825154 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:22.839086 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efad4fd8-e587-41e8-8d0c-48b4b1f8ca57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:22.840550 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efad4fd8-e587-41e8-8d0c-48b4b1f8ca57 service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:22.852884 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1260/2529] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:22:26.538899 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2465ebc-6cc7-462e-ac5c-ecaf591478a3 service cinder] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:26.548636 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2465ebc-6cc7-462e-ac5c-ecaf591478a3 service cinder] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:26.553334 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c985b-eb4a-4834-8f23-1ce28bffb14b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:26.554008 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c985b-eb4a-4834-8f23-1ce28bffb14b service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:26.554502 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c985b-eb4a-4834-8f23-1ce28bffb14b service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:26.554789 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c985b-eb4a-4834-8f23-1ce28bffb14b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:26.566368 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c985b-eb4a-4834-8f23-1ce28bffb14b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:26.567853 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c985b-eb4a-4834-8f23-1ce28bffb14b service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:26.579692 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1270/2530] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:26 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:22:26.949573 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efad4fd8-e587-41e8-8d0c-48b4b1f8ca57 service cinder] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:26.959232 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efad4fd8-e587-41e8-8d0c-48b4b1f8ca57 service cinder] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:26.962994 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c208e1b-275e-408a-b39c-1aec7fa7588e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:26.963641 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c208e1b-275e-408a-b39c-1aec7fa7588e service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:26.963908 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c208e1b-275e-408a-b39c-1aec7fa7588e service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:26.964312 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c208e1b-275e-408a-b39c-1aec7fa7588e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:27.026455 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c208e1b-275e-408a-b39c-1aec7fa7588e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:27.028193 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c208e1b-275e-408a-b39c-1aec7fa7588e service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:27.038267 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1261/2531] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:22:29.173775 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c985b-eb4a-4834-8f23-1ce28bffb14b service cinder] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:29.259165 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c985b-eb4a-4834-8f23-1ce28bffb14b service cinder] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:29.266586 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a42d427-7f20-4a9b-8957-2508c9930e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:29.267474 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42d427-7f20-4a9b-8957-2508c9930e1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:29.267757 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42d427-7f20-4a9b-8957-2508c9930e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:29.268029 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42d427-7f20-4a9b-8957-2508c9930e1d admin admin] PATH_INFO: `/v3/users/000ca70757c943a58329416fe4bcd94a` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:29.269288 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a42d427-7f20-4a9b-8957-2508c9930e1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=000ca70757c943a58329416fe4bcd94a)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:29.271234 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a42d427-7f20-4a9b-8957-2508c9930e1d admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:29.296054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:29.329539 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8a42d427-7f20-4a9b-8957-2508c9930e1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '000ca70757c943a58329416fe4bcd94a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:29.337427 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8a42d427-7f20-4a9b-8957-2508c9930e1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '000ca70757c943a58329416fe4bcd94a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:29.338255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8a42d427-7f20-4a9b-8957-2508c9930e1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '000ca70757c943a58329416fe4bcd94a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:29.338858 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8a42d427-7f20-4a9b-8957-2508c9930e1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '000ca70757c943a58329416fe4bcd94a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:29.348088 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8a42d427-7f20-4a9b-8957-2508c9930e1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '000ca70757c943a58329416fe4bcd94a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:29.367776 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1271/2532] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:22:29 2022] DELETE /identity/v3/users/000ca70757c943a58329416fe4bcd94a => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:29.372469 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042df165-9f23-4825-a1c2-90d20b2bd51a None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:29.373010 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042df165-9f23-4825-a1c2-90d20b2bd51a None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:29.373112 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042df165-9f23-4825-a1c2-90d20b2bd51a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:29.391338 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-042df165-9f23-4825-a1c2-90d20b2bd51a None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:29.392549 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-042df165-9f23-4825-a1c2-90d20b2bd51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:29.462888 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1262/2533] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:29.477225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a42d427-7f20-4a9b-8957-2508c9930e1d admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:29.478842 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-8a42d427-7f20-4a9b-8957-2508c9930e1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:29.548995 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a42d427-7f20-4a9b-8957-2508c9930e1d admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:29.552842 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d50aec-bbea-40b5-9e0f-80fdfabcc8c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:29.553678 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d50aec-bbea-40b5-9e0f-80fdfabcc8c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:29.554215 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d50aec-bbea-40b5-9e0f-80fdfabcc8c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:29.554641 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d50aec-bbea-40b5-9e0f-80fdfabcc8c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:29.572132 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d50aec-bbea-40b5-9e0f-80fdfabcc8c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:29.574989 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d50aec-bbea-40b5-9e0f-80fdfabcc8c2 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:29.588358 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1272/2534] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:22:29.880403 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042df165-9f23-4825-a1c2-90d20b2bd51a None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:29.889742 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-042df165-9f23-4825-a1c2-90d20b2bd51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:29.903807 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042df165-9f23-4825-a1c2-90d20b2bd51a None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:29.909205 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad285c6-4fef-4c81-8372-77c279cea56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:29.910774 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad285c6-4fef-4c81-8372-77c279cea56c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:29.913000 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad285c6-4fef-4c81-8372-77c279cea56c admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:29.913000 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad285c6-4fef-4c81-8372-77c279cea56c admin admin] PATH_INFO: `/v3/projects/a31610abf8444a3fb4b0cc98489915be` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:29.913000 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad285c6-4fef-4c81-8372-77c279cea56c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a31610abf8444a3fb4b0cc98489915be)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:29.915802 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad285c6-4fef-4c81-8372-77c279cea56c admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:29.982782 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-2ad285c6-4fef-4c81-8372-77c279cea56c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a31610abf8444a3fb4b0cc98489915be', 'request_id': 'req-2ad285c6-4fef-4c81-8372-77c279cea56c'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:29.989371 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-2ad285c6-4fef-4c81-8372-77c279cea56c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a31610abf8444a3fb4b0cc98489915be', 'request_id': 'req-2ad285c6-4fef-4c81-8372-77c279cea56c'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:29.999159 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-2ad285c6-4fef-4c81-8372-77c279cea56c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a31610abf8444a3fb4b0cc98489915be', 'request_id': 'req-2ad285c6-4fef-4c81-8372-77c279cea56c'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:30.012624 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-2ad285c6-4fef-4c81-8372-77c279cea56c admin admin] The token cache is being invalidate because project a31610abf8444a3fb4b0cc98489915be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:22:30.012966 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-2ad285c6-4fef-4c81-8372-77c279cea56c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:30.013761 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-2ad285c6-4fef-4c81-8372-77c279cea56c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:30.015817 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1263/2535] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:22:29 2022] DELETE /identity/v3/projects/a31610abf8444a3fb4b0cc98489915be => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:30.020610 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d50aec-bbea-40b5-9e0f-80fdfabcc8c2 service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:30.040095 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d50aec-bbea-40b5-9e0f-80fdfabcc8c2 service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:30.104099 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b55abdf-b067-4c27-84e3-06b319e1cd18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:30.104946 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55abdf-b067-4c27-84e3-06b319e1cd18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:30.105247 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55abdf-b067-4c27-84e3-06b319e1cd18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:30.105525 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55abdf-b067-4c27-84e3-06b319e1cd18 admin admin] PATH_INFO: `/v3/users/bd52c01f43794b7080fe736d6db70260` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:30.106779 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b55abdf-b067-4c27-84e3-06b319e1cd18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd52c01f43794b7080fe736d6db70260)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:30.108740 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b55abdf-b067-4c27-84e3-06b319e1cd18 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:30.157197 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:30.159274 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-4b55abdf-b067-4c27-84e3-06b319e1cd18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd52c01f43794b7080fe736d6db70260'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:30.168316 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-4b55abdf-b067-4c27-84e3-06b319e1cd18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd52c01f43794b7080fe736d6db70260'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:30.169102 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-4b55abdf-b067-4c27-84e3-06b319e1cd18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd52c01f43794b7080fe736d6db70260'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:30.169786 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-4b55abdf-b067-4c27-84e3-06b319e1cd18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd52c01f43794b7080fe736d6db70260'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:30.178364 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-4b55abdf-b067-4c27-84e3-06b319e1cd18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd52c01f43794b7080fe736d6db70260'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:30.205619 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1273/2536] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:22:30 2022] DELETE /identity/v3/users/bd52c01f43794b7080fe736d6db70260 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:30.221934 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad285c6-4fef-4c81-8372-77c279cea56c admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:30.223410 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-2ad285c6-4fef-4c81-8372-77c279cea56c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:30.234498 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:30.291480 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad285c6-4fef-4c81-8372-77c279cea56c admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:30.344768 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc990c28-a5d7-4df7-94e2-ba93e7b61911 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:30.345465 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc990c28-a5d7-4df7-94e2-ba93e7b61911 service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:30.345753 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc990c28-a5d7-4df7-94e2-ba93e7b61911 service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:30.346178 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc990c28-a5d7-4df7-94e2-ba93e7b61911 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:30.348014 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-bc990c28-a5d7-4df7-94e2-ba93e7b61911 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:30.433352 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc990c28-a5d7-4df7-94e2-ba93e7b61911 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:30.434922 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc990c28-a5d7-4df7-94e2-ba93e7b61911 service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:30.448868 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1264/2537] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:30 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:22:30.585704 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b55abdf-b067-4c27-84e3-06b319e1cd18 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:30.587652 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-4b55abdf-b067-4c27-84e3-06b319e1cd18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:30.665279 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b55abdf-b067-4c27-84e3-06b319e1cd18 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:30.670371 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d79f1-a458-42b2-9484-2c4623ecc6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:30.671314 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d79f1-a458-42b2-9484-2c4623ecc6cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:30.671597 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d79f1-a458-42b2-9484-2c4623ecc6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:30.671874 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d79f1-a458-42b2-9484-2c4623ecc6cd admin admin] PATH_INFO: `/v3/projects/fd50724cd4e64e5f826bd8afc44a4808` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:30.673219 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d79f1-a458-42b2-9484-2c4623ecc6cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd50724cd4e64e5f826bd8afc44a4808)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:30.675350 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d79f1-a458-42b2-9484-2c4623ecc6cd admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:30.728699 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-887d79f1-a458-42b2-9484-2c4623ecc6cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd50724cd4e64e5f826bd8afc44a4808', 'request_id': 'req-887d79f1-a458-42b2-9484-2c4623ecc6cd'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:30.735823 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-887d79f1-a458-42b2-9484-2c4623ecc6cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd50724cd4e64e5f826bd8afc44a4808', 'request_id': 'req-887d79f1-a458-42b2-9484-2c4623ecc6cd'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:30.746940 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-887d79f1-a458-42b2-9484-2c4623ecc6cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd50724cd4e64e5f826bd8afc44a4808', 'request_id': 'req-887d79f1-a458-42b2-9484-2c4623ecc6cd'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:30.758625 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-887d79f1-a458-42b2-9484-2c4623ecc6cd admin admin] The token cache is being invalidate because project fd50724cd4e64e5f826bd8afc44a4808 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:22:30.758906 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-887d79f1-a458-42b2-9484-2c4623ecc6cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:30.759634 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-887d79f1-a458-42b2-9484-2c4623ecc6cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:30.761890 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1274/2538] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:22:30 2022] DELETE /identity/v3/projects/fd50724cd4e64e5f826bd8afc44a4808 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:30.765852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc990c28-a5d7-4df7-94e2-ba93e7b61911 service neutron] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:30.777278 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc990c28-a5d7-4df7-94e2-ba93e7b61911 service neutron] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:30.847085 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dcae7e-d876-466e-a09f-fae68941873b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:30.847988 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dcae7e-d876-466e-a09f-fae68941873b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:30.848264 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dcae7e-d876-466e-a09f-fae68941873b admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:30.848542 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dcae7e-d876-466e-a09f-fae68941873b admin admin] PATH_INFO: `/v3/users/2125b3117ba04f88a891beace8ba8c23` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:30.850189 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dcae7e-d876-466e-a09f-fae68941873b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2125b3117ba04f88a891beace8ba8c23)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:30.852261 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dcae7e-d876-466e-a09f-fae68941873b admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:30.870627 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:30.905631 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-38dcae7e-d876-466e-a09f-fae68941873b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2125b3117ba04f88a891beace8ba8c23'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:30.915124 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-38dcae7e-d876-466e-a09f-fae68941873b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2125b3117ba04f88a891beace8ba8c23'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:30.915888 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-38dcae7e-d876-466e-a09f-fae68941873b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2125b3117ba04f88a891beace8ba8c23'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:30.916492 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-38dcae7e-d876-466e-a09f-fae68941873b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2125b3117ba04f88a891beace8ba8c23'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:30.928053 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-38dcae7e-d876-466e-a09f-fae68941873b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2125b3117ba04f88a891beace8ba8c23'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:30.950846 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1265/2539] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:22:30 2022] DELETE /identity/v3/users/2125b3117ba04f88a891beace8ba8c23 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:22:31.274600 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d79f1-a458-42b2-9484-2c4623ecc6cd admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:31.276350 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-887d79f1-a458-42b2-9484-2c4623ecc6cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:31.302715 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d79f1-a458-42b2-9484-2c4623ecc6cd admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:31.366654 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:31.380648 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c368b-5349-4629-a30c-209ea58046ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:31.382030 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c368b-5349-4629-a30c-209ea58046ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:31.383415 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c368b-5349-4629-a30c-209ea58046ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:31.383816 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c368b-5349-4629-a30c-209ea58046ea admin admin] PATH_INFO: `/v3/projects/a23d6eda15a7404d9259bb0e176d81a7` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:31.385378 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5c368b-5349-4629-a30c-209ea58046ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=a23d6eda15a7404d9259bb0e176d81a7)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:31.387457 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5c368b-5349-4629-a30c-209ea58046ea admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:31.449061 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8e5c368b-5349-4629-a30c-209ea58046ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a23d6eda15a7404d9259bb0e176d81a7', 'request_id': 'req-8e5c368b-5349-4629-a30c-209ea58046ea'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:31.461235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8e5c368b-5349-4629-a30c-209ea58046ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a23d6eda15a7404d9259bb0e176d81a7', 'request_id': 'req-8e5c368b-5349-4629-a30c-209ea58046ea'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:31.473816 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8e5c368b-5349-4629-a30c-209ea58046ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a23d6eda15a7404d9259bb0e176d81a7', 'request_id': 'req-8e5c368b-5349-4629-a30c-209ea58046ea'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:31.487202 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8e5c368b-5349-4629-a30c-209ea58046ea admin admin] The token cache is being invalidate because project a23d6eda15a7404d9259bb0e176d81a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:22:31.487488 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8e5c368b-5349-4629-a30c-209ea58046ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:31.488146 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8e5c368b-5349-4629-a30c-209ea58046ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:31.490094 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1275/2540] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:22:31 2022] DELETE /identity/v3/projects/a23d6eda15a7404d9259bb0e176d81a7 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:31.496093 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9991e55-0dca-48f6-9343-b4b32a026078 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:31.496717 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9991e55-0dca-48f6-9343-b4b32a026078 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:31.496717 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9991e55-0dca-48f6-9343-b4b32a026078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:31.515973 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-f9991e55-0dca-48f6-9343-b4b32a026078 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:31.517433 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-f9991e55-0dca-48f6-9343-b4b32a026078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:31.562452 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c368b-5349-4629-a30c-209ea58046ea admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:31.564383 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-8e5c368b-5349-4629-a30c-209ea58046ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:31.596974 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1266/2541] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:31.599993 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8db8fb-ddc4-497a-b56b-7c090a2c08d2 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:31.600278 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8db8fb-ddc4-497a-b56b-7c090a2c08d2 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:31.600552 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8db8fb-ddc4-497a-b56b-7c090a2c08d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:31.621645 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-ad8db8fb-ddc4-497a-b56b-7c090a2c08d2 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:31.623009 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-ad8db8fb-ddc4-497a-b56b-7c090a2c08d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:31.631729 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1267/2542] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:22:31.635938 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8db8fb-ddc4-497a-b56b-7c090a2c08d2 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:31.647317 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8db8fb-ddc4-497a-b56b-7c090a2c08d2 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:31.653058 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b5dfc-e9a0-40b7-b094-d2445bf6562f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:31.654110 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b5dfc-e9a0-40b7-b094-d2445bf6562f admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:31.654110 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b5dfc-e9a0-40b7-b094-d2445bf6562f admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:31.654359 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c368b-5349-4629-a30c-209ea58046ea admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:31.654467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b5dfc-e9a0-40b7-b094-d2445bf6562f admin admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:31.654993 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223b5dfc-e9a0-40b7-b094-d2445bf6562f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:22:31.655315 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223b5dfc-e9a0-40b7-b094-d2445bf6562f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:31.656984 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223b5dfc-e9a0-40b7-b094-d2445bf6562f admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:31.659882 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc8914c-4f2a-4ee2-b884-e89fe0f83c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:31.660869 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc8914c-4f2a-4ee2-b884-e89fe0f83c70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:31.661279 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc8914c-4f2a-4ee2-b884-e89fe0f83c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:31.661654 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc8914c-4f2a-4ee2-b884-e89fe0f83c70 admin admin] PATH_INFO: `/v3/users/93eb53f596674fa387c1ce83f898d05a` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:31.663005 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc8914c-4f2a-4ee2-b884-e89fe0f83c70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93eb53f596674fa387c1ce83f898d05a)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:31.665181 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc8914c-4f2a-4ee2-b884-e89fe0f83c70 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:31.672210 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1268/2543] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:22:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:22:31.675997 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b5dfc-e9a0-40b7-b094-d2445bf6562f admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:31.686903 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b5dfc-e9a0-40b7-b094-d2445bf6562f admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:31.692653 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85982a54-a0ff-4172-b825-a13a3cb97e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:31.693491 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85982a54-a0ff-4172-b825-a13a3cb97e50 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:31.693775 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85982a54-a0ff-4172-b825-a13a3cb97e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:31.694047 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85982a54-a0ff-4172-b825-a13a3cb97e50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:31.694805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85982a54-a0ff-4172-b825-a13a3cb97e50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:31.696913 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85982a54-a0ff-4172-b825-a13a3cb97e50 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:31.728725 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1269/2544] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:22:31 2022] POST /identity/v3/projects => generated 367 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:22:31.733310 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85982a54-a0ff-4172-b825-a13a3cb97e50 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:31.750771 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8dc8914c-4f2a-4ee2-b884-e89fe0f83c70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93eb53f596674fa387c1ce83f898d05a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:31.761864 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8dc8914c-4f2a-4ee2-b884-e89fe0f83c70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93eb53f596674fa387c1ce83f898d05a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:31.762919 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8dc8914c-4f2a-4ee2-b884-e89fe0f83c70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93eb53f596674fa387c1ce83f898d05a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:31.763702 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8dc8914c-4f2a-4ee2-b884-e89fe0f83c70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93eb53f596674fa387c1ce83f898d05a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:31.777889 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8dc8914c-4f2a-4ee2-b884-e89fe0f83c70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93eb53f596674fa387c1ce83f898d05a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:31.800688 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:31.808494 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1276/2545] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:22:31 2022] DELETE /identity/v3/users/93eb53f596674fa387c1ce83f898d05a => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:31.810111 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101848c-181b-4e0f-b1b1-9a47bfa8b962 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:31.810421 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101848c-181b-4e0f-b1b1-9a47bfa8b962 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:31.810699 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101848c-181b-4e0f-b1b1-9a47bfa8b962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:31.814936 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85982a54-a0ff-4172-b825-a13a3cb97e50 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:31.818073 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:31.833003 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-e101848c-181b-4e0f-b1b1-9a47bfa8b962 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:31.834573 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-e101848c-181b-4e0f-b1b1-9a47bfa8b962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:31.888127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761857b2-27b7-4395-bb4b-c9ab8a84b8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:31.888807 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761857b2-27b7-4395-bb4b-c9ab8a84b8b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:31.889186 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761857b2-27b7-4395-bb4b-c9ab8a84b8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:31.889526 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761857b2-27b7-4395-bb4b-c9ab8a84b8b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:31.890479 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761857b2-27b7-4395-bb4b-c9ab8a84b8b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:31.892449 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761857b2-27b7-4395-bb4b-c9ab8a84b8b5 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:31.906165 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1277/2546] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:22:31.916997 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e101848c-181b-4e0f-b1b1-9a47bfa8b962 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:31.919121 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-e101848c-181b-4e0f-b1b1-9a47bfa8b962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:31.919516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1270/2547] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:22:31 2022] POST /identity/v3/users => generated 284 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:22:31.923287 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761857b2-27b7-4395-bb4b-c9ab8a84b8b5 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:31.925869 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-761857b2-27b7-4395-bb4b-c9ab8a84b8b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:31.943081 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761857b2-27b7-4395-bb4b-c9ab8a84b8b5 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:31.948565 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab1fbdc-851a-4766-92cd-f08e531379a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:31.949522 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1fbdc-851a-4766-92cd-f08e531379a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:31.949790 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1fbdc-851a-4766-92cd-f08e531379a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:31.950165 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1fbdc-851a-4766-92cd-f08e531379a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:31.951032 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab1fbdc-851a-4766-92cd-f08e531379a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:31.953505 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab1fbdc-851a-4766-92cd-f08e531379a8 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:31.972613 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1271/2548] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:31 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:22:31.974264 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab1fbdc-851a-4766-92cd-f08e531379a8 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:31.985363 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab1fbdc-851a-4766-92cd-f08e531379a8 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:31.990092 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e101848c-181b-4e0f-b1b1-9a47bfa8b962 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:31.990955 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb2ec74-0602-49d0-93da-0a20dae9551a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:31.991321 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2ec74-0602-49d0-93da-0a20dae9551a admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:31.991943 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2ec74-0602-49d0-93da-0a20dae9551a admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:31.992874 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2ec74-0602-49d0-93da-0a20dae9551a admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:31.993335 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb2ec74-0602-49d0-93da-0a20dae9551a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:31.995046 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb2ec74-0602-49d0-93da-0a20dae9551a admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:31.995585 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30843596-3d69-4677-a1bd-cede79a0bff0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:31.996250 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30843596-3d69-4677-a1bd-cede79a0bff0 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:31.996570 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30843596-3d69-4677-a1bd-cede79a0bff0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:31.996800 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30843596-3d69-4677-a1bd-cede79a0bff0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:32.008737 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1272/2549] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:22:32.012530 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb2ec74-0602-49d0-93da-0a20dae9551a admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:32.014030 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30843596-3d69-4677-a1bd-cede79a0bff0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:32.015538 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30843596-3d69-4677-a1bd-cede79a0bff0 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:32.028641 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1278/2550] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:22:32.033939 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb2ec74-0602-49d0-93da-0a20dae9551a admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:32.039813 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a2e80-8885-4581-a4bd-d17b21802dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:32.040760 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a2e80-8885-4581-a4bd-d17b21802dfc admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:32.041084 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a2e80-8885-4581-a4bd-d17b21802dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:32.041361 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765a2e80-8885-4581-a4bd-d17b21802dfc admin admin] PATH_INFO: `/v3/projects/b609118f5342413b8ec541b95dbe8d27/users/60794b2c0e734b7b99cd66d72087a6c6/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:32.059755 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765a2e80-8885-4581-a4bd-d17b21802dfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b609118f5342413b8ec541b95dbe8d27, user_id=60794b2c0e734b7b99cd66d72087a6c6, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:32.062516 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765a2e80-8885-4581-a4bd-d17b21802dfc admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:32.078746 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1273/2551] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:22:32 2022] PUT /identity/v3/projects/b609118f5342413b8ec541b95dbe8d27/users/60794b2c0e734b7b99cd66d72087a6c6/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:22:32.090919 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a045da6-2f84-43f8-8cfe-6dfdd28bec16 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:32.091219 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a045da6-2f84-43f8-8cfe-6dfdd28bec16 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:32.091544 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a045da6-2f84-43f8-8cfe-6dfdd28bec16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:32.125828 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-0a045da6-2f84-43f8-8cfe-6dfdd28bec16 None None] MFA Rules not processed for user `60794b2c0e734b7b99cd66d72087a6c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:32.127325 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-0a045da6-2f84-43f8-8cfe-6dfdd28bec16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:32.231705 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1279/2552] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:32 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:32.236513 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a2e80-8885-4581-a4bd-d17b21802dfc admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:32.314539 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765a2e80-8885-4581-a4bd-d17b21802dfc admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:32.319592 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc178ab-5b8c-4b2f-9e37-31f75a09ba87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:32.320289 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc178ab-5b8c-4b2f-9e37-31f75a09ba87 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:32.320562 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc178ab-5b8c-4b2f-9e37-31f75a09ba87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:32.320830 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc178ab-5b8c-4b2f-9e37-31f75a09ba87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:32.321558 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc178ab-5b8c-4b2f-9e37-31f75a09ba87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:32.323520 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc178ab-5b8c-4b2f-9e37-31f75a09ba87 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:32.347644 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a045da6-2f84-43f8-8cfe-6dfdd28bec16 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:32.349682 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-0a045da6-2f84-43f8-8cfe-6dfdd28bec16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:32.368487 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a045da6-2f84-43f8-8cfe-6dfdd28bec16 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:32.375082 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d2bc0-6b41-4f0b-8f5d-77407f8559f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:32.375970 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d2bc0-6b41-4f0b-8f5d-77407f8559f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:32.376405 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d2bc0-6b41-4f0b-8f5d-77407f8559f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:32.376521 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d2bc0-6b41-4f0b-8f5d-77407f8559f3 admin admin] PATH_INFO: `/v3/projects/c53f85f9a90c4751b5f5b5412dddc507` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:32.376694 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1274/2553] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:22:32 2022] POST /identity/v3/projects => generated 367 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:22:32.377956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354d2bc0-6b41-4f0b-8f5d-77407f8559f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c53f85f9a90c4751b5f5b5412dddc507)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:32.380143 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc178ab-5b8c-4b2f-9e37-31f75a09ba87 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:32.381458 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354d2bc0-6b41-4f0b-8f5d-77407f8559f3 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:32.421239 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:32.422166 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:32.441012 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-354d2bc0-6b41-4f0b-8f5d-77407f8559f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c53f85f9a90c4751b5f5b5412dddc507', 'request_id': 'req-354d2bc0-6b41-4f0b-8f5d-77407f8559f3'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:32.448473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-354d2bc0-6b41-4f0b-8f5d-77407f8559f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c53f85f9a90c4751b5f5b5412dddc507', 'request_id': 'req-354d2bc0-6b41-4f0b-8f5d-77407f8559f3'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:32.458233 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-354d2bc0-6b41-4f0b-8f5d-77407f8559f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c53f85f9a90c4751b5f5b5412dddc507', 'request_id': 'req-354d2bc0-6b41-4f0b-8f5d-77407f8559f3'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:32.465311 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc178ab-5b8c-4b2f-9e37-31f75a09ba87 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:32.473173 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-354d2bc0-6b41-4f0b-8f5d-77407f8559f3 admin admin] The token cache is being invalidate because project c53f85f9a90c4751b5f5b5412dddc507 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:22:32.473507 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-354d2bc0-6b41-4f0b-8f5d-77407f8559f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:32.474614 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-354d2bc0-6b41-4f0b-8f5d-77407f8559f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:32.479034 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1280/2554] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:22:32 2022] DELETE /identity/v3/projects/c53f85f9a90c4751b5f5b5412dddc507 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:32.480116 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d2bc0-6b41-4f0b-8f5d-77407f8559f3 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:32.481673 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-354d2bc0-6b41-4f0b-8f5d-77407f8559f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:32.554746 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a05dcf-e4e5-41d0-994d-e800d443e982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:32.555446 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a05dcf-e4e5-41d0-994d-e800d443e982 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:32.555731 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a05dcf-e4e5-41d0-994d-e800d443e982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:32.556010 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a05dcf-e4e5-41d0-994d-e800d443e982 admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:32.556712 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a05dcf-e4e5-41d0-994d-e800d443e982 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:32.559953 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a05dcf-e4e5-41d0-994d-e800d443e982 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:32.587171 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1275/2555] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:22:32 2022] POST /identity/v3/users => generated 290 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:22:32.587442 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d2bc0-6b41-4f0b-8f5d-77407f8559f3 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:32.588095 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a05dcf-e4e5-41d0-994d-e800d443e982 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:32.589779 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-04a05dcf-e4e5-41d0-994d-e800d443e982 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:32.594294 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d56bdde-7c17-4dc8-bb51-828b93ce7adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:32.595486 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56bdde-7c17-4dc8-bb51-828b93ce7adc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:32.596484 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56bdde-7c17-4dc8-bb51-828b93ce7adc admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:32.596484 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56bdde-7c17-4dc8-bb51-828b93ce7adc admin admin] PATH_INFO: `/v3/users/a83cc7cd946946bbb355c217e6256d9e` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:32.597488 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d56bdde-7c17-4dc8-bb51-828b93ce7adc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a83cc7cd946946bbb355c217e6256d9e)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:32.599710 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d56bdde-7c17-4dc8-bb51-828b93ce7adc admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:32.654525 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-7d56bdde-7c17-4dc8-bb51-828b93ce7adc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a83cc7cd946946bbb355c217e6256d9e'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:32.663200 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-7d56bdde-7c17-4dc8-bb51-828b93ce7adc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a83cc7cd946946bbb355c217e6256d9e'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:32.664631 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-7d56bdde-7c17-4dc8-bb51-828b93ce7adc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a83cc7cd946946bbb355c217e6256d9e'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:32.665380 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-7d56bdde-7c17-4dc8-bb51-828b93ce7adc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a83cc7cd946946bbb355c217e6256d9e'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:32.674170 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-7d56bdde-7c17-4dc8-bb51-828b93ce7adc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a83cc7cd946946bbb355c217e6256d9e'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:32.681590 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a05dcf-e4e5-41d0-994d-e800d443e982 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:32.687544 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9787029-762a-4c8d-b354-ef2fa26a95e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:32.688230 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9787029-762a-4c8d-b354-ef2fa26a95e3 service nova] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:32.688512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9787029-762a-4c8d-b354-ef2fa26a95e3 service nova] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:32.688783 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9787029-762a-4c8d-b354-ef2fa26a95e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:32.690788 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-e9787029-762a-4c8d-b354-ef2fa26a95e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:32.696888 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1281/2556] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:22:32 2022] DELETE /identity/v3/users/a83cc7cd946946bbb355c217e6256d9e => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:32.700187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d56bdde-7c17-4dc8-bb51-828b93ce7adc admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:32.704985 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9787029-762a-4c8d-b354-ef2fa26a95e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:32.706658 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9787029-762a-4c8d-b354-ef2fa26a95e3 service nova] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:32.707112 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-7d56bdde-7c17-4dc8-bb51-828b93ce7adc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:32.720576 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1276/2557] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:22:32.778107 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:32.816690 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d56bdde-7c17-4dc8-bb51-828b93ce7adc admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:32.822830 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb5f975-18ef-4c1c-acb6-0b2a6c4eaf9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:32.824067 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5f975-18ef-4c1c-acb6-0b2a6c4eaf9c admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:32.824272 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5f975-18ef-4c1c-acb6-0b2a6c4eaf9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:32.824645 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5f975-18ef-4c1c-acb6-0b2a6c4eaf9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:32.825680 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb5f975-18ef-4c1c-acb6-0b2a6c4eaf9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:32.828054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb5f975-18ef-4c1c-acb6-0b2a6c4eaf9c admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:32.857802 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1282/2558] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:32 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:22:32.859400 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9787029-762a-4c8d-b354-ef2fa26a95e3 service nova] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:32.874661 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9787029-762a-4c8d-b354-ef2fa26a95e3 service nova] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:32.889220 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d1c3a-f862-4951-8a70-24d64d86916d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:32.890149 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d1c3a-f862-4951-8a70-24d64d86916d admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:32.890436 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d1c3a-f862-4951-8a70-24d64d86916d admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:32.890733 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d1c3a-f862-4951-8a70-24d64d86916d admin admin] PATH_INFO: `/v3/projects/cf650ec2097548e0a50e2d68f58cb777/users/d2fb6e6f971247afb7ea0972f2c8d914/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:32.922863 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3d1c3a-f862-4951-8a70-24d64d86916d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf650ec2097548e0a50e2d68f58cb777, user_id=d2fb6e6f971247afb7ea0972f2c8d914, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:32.926411 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3d1c3a-f862-4951-8a70-24d64d86916d admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:32.966954 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1277/2559] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:22:32 2022] PUT /identity/v3/projects/cf650ec2097548e0a50e2d68f58cb777/users/d2fb6e6f971247afb7ea0972f2c8d914/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:32.968762 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb5f975-18ef-4c1c-acb6-0b2a6c4eaf9c admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:32.990582 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb5f975-18ef-4c1c-acb6-0b2a6c4eaf9c admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:33.094893 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de9842-9aea-40d7-b2ba-5760b04c63b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:33.096163 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de9842-9aea-40d7-b2ba-5760b04c63b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:33.096436 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d1c3a-f862-4951-8a70-24d64d86916d admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:33.097887 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de9842-9aea-40d7-b2ba-5760b04c63b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:33.098176 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de9842-9aea-40d7-b2ba-5760b04c63b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:33.098428 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-9e3d1c3a-f862-4951-8a70-24d64d86916d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:33.098937 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de9842-9aea-40d7-b2ba-5760b04c63b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:33.103280 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de9842-9aea-40d7-b2ba-5760b04c63b4 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:33.123702 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1283/2560] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:32 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:22:33.124636 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d1c3a-f862-4951-8a70-24d64d86916d admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:33.125156 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de9842-9aea-40d7-b2ba-5760b04c63b4 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:33.127292 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2778895-bc9f-456b-9988-ae936e53c811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:33.127906 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2778895-bc9f-456b-9988-ae936e53c811 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:33.128359 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2778895-bc9f-456b-9988-ae936e53c811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:33.128475 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2778895-bc9f-456b-9988-ae936e53c811 admin admin] PATH_INFO: `/v3/projects/8ec80a93546e4ca3a9222562c0423da6` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:33.129685 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2778895-bc9f-456b-9988-ae936e53c811 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ec80a93546e4ca3a9222562c0423da6)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:33.131975 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2778895-bc9f-456b-9988-ae936e53c811 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:33.146960 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de9842-9aea-40d7-b2ba-5760b04c63b4 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:33.158356 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734f8774-b2ae-4986-ad4d-d31558ade517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:33.159285 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734f8774-b2ae-4986-ad4d-d31558ade517 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:33.159570 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734f8774-b2ae-4986-ad4d-d31558ade517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:33.159835 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734f8774-b2ae-4986-ad4d-d31558ade517 admin admin] PATH_INFO: `/v3/projects/cf650ec2097548e0a50e2d68f58cb777/users/d2fb6e6f971247afb7ea0972f2c8d914/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:33.164458 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734f8774-b2ae-4986-ad4d-d31558ade517 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf650ec2097548e0a50e2d68f58cb777, user_id=d2fb6e6f971247afb7ea0972f2c8d914, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:33.165898 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:33.169616 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734f8774-b2ae-4986-ad4d-d31558ade517 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:33.198699 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1284/2561] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:22:33 2022] PUT /identity/v3/projects/cf650ec2097548e0a50e2d68f58cb777/users/d2fb6e6f971247afb7ea0972f2c8d914/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:33.215769 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ccecdd-d6ef-4d62-ad43-cec5424c6a47 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:33.216084 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ccecdd-d6ef-4d62-ad43-cec5424c6a47 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:33.216367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ccecdd-d6ef-4d62-ad43-cec5424c6a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:33.217419 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-b2778895-bc9f-456b-9988-ae936e53c811 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ec80a93546e4ca3a9222562c0423da6', 'request_id': 'req-b2778895-bc9f-456b-9988-ae936e53c811'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:33.238016 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-b2778895-bc9f-456b-9988-ae936e53c811 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ec80a93546e4ca3a9222562c0423da6', 'request_id': 'req-b2778895-bc9f-456b-9988-ae936e53c811'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:33.271679 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-b2778895-bc9f-456b-9988-ae936e53c811 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ec80a93546e4ca3a9222562c0423da6', 'request_id': 'req-b2778895-bc9f-456b-9988-ae936e53c811'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:33.290863 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-02ccecdd-d6ef-4d62-ad43-cec5424c6a47 None None] MFA Rules not processed for user `d2fb6e6f971247afb7ea0972f2c8d914`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:33.292479 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-02ccecdd-d6ef-4d62-ad43-cec5424c6a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:33.307021 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-b2778895-bc9f-456b-9988-ae936e53c811 admin admin] The token cache is being invalidate because project 8ec80a93546e4ca3a9222562c0423da6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:22:33.307630 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-b2778895-bc9f-456b-9988-ae936e53c811 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:33.308514 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-b2778895-bc9f-456b-9988-ae936e53c811 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:33.315294 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1278/2562] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:22:33 2022] DELETE /identity/v3/projects/8ec80a93546e4ca3a9222562c0423da6 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:33.322818 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c8e78f-5d41-474b-9847-00a5b69f9152 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:33.323558 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c8e78f-5d41-474b-9847-00a5b69f9152 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:33.323967 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c8e78f-5d41-474b-9847-00a5b69f9152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:33.378215 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-b5c8e78f-5d41-474b-9847-00a5b69f9152 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:33.379482 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-b5c8e78f-5d41-474b-9847-00a5b69f9152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:33.405397 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1285/2563] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:33 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:22:33.418432 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ccecdd-d6ef-4d62-ad43-cec5424c6a47 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:33.420550 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-02ccecdd-d6ef-4d62-ad43-cec5424c6a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:33.479259 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:33.492381 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1279/2564] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:33.502933 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ccecdd-d6ef-4d62-ad43-cec5424c6a47 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:33.503448 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031772dd-558e-4e09-9135-1af3bcd78dd0 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:33.503744 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031772dd-558e-4e09-9135-1af3bcd78dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:33.504025 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031772dd-558e-4e09-9135-1af3bcd78dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:33.508786 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79966b1a-279b-49ca-beda-8593bff367fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:33.511069 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79966b1a-279b-49ca-beda-8593bff367fc service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:33.515507 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79966b1a-279b-49ca-beda-8593bff367fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:33.515940 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79966b1a-279b-49ca-beda-8593bff367fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:33.529179 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79966b1a-279b-49ca-beda-8593bff367fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:33.531093 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-031772dd-558e-4e09-9135-1af3bcd78dd0 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:33.531383 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79966b1a-279b-49ca-beda-8593bff367fc service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:33.532854 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-031772dd-558e-4e09-9135-1af3bcd78dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:33.543245 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1286/2566] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:33 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:22:33.543606 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1280/2566] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:33.568774 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79966b1a-279b-49ca-beda-8593bff367fc service cinder] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:33.598681 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79966b1a-279b-49ca-beda-8593bff367fc service cinder] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:33.604713 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e4479c-7add-4835-9b3b-7281299cdbaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:33.605447 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4479c-7add-4835-9b3b-7281299cdbaa admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:33.605711 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4479c-7add-4835-9b3b-7281299cdbaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:33.606015 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4479c-7add-4835-9b3b-7281299cdbaa admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:33.607146 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e4479c-7add-4835-9b3b-7281299cdbaa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:22:33.607146 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e4479c-7add-4835-9b3b-7281299cdbaa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:33.608619 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e4479c-7add-4835-9b3b-7281299cdbaa admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:33.624883 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1287/2567] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:22:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:22:33.630112 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031772dd-558e-4e09-9135-1af3bcd78dd0 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:33.640816 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031772dd-558e-4e09-9135-1af3bcd78dd0 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:33.646060 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f5e0b4-5d01-4cb5-9a92-b08214be900f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:33.646807 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5e0b4-5d01-4cb5-9a92-b08214be900f admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:33.647094 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5e0b4-5d01-4cb5-9a92-b08214be900f admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:33.647369 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5e0b4-5d01-4cb5-9a92-b08214be900f admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:33.648386 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f5e0b4-5d01-4cb5-9a92-b08214be900f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:33.653518 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e4479c-7add-4835-9b3b-7281299cdbaa admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:33.654031 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f5e0b4-5d01-4cb5-9a92-b08214be900f admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:33.666243 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e4479c-7add-4835-9b3b-7281299cdbaa admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:33.671477 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c548f375-d384-407b-a9ec-a050049acf07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:33.672220 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548f375-d384-407b-a9ec-a050049acf07 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:33.672566 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548f375-d384-407b-a9ec-a050049acf07 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:33.672905 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548f375-d384-407b-a9ec-a050049acf07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:33.674852 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-c548f375-d384-407b-a9ec-a050049acf07 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:33.693458 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1281/2568] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:22:33 2022] POST /identity/v3/projects => generated 373 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:22:33.694534 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f5e0b4-5d01-4cb5-9a92-b08214be900f admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:33.760884 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c548f375-d384-407b-a9ec-a050049acf07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:33.763334 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c548f375-d384-407b-a9ec-a050049acf07 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:33.775467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f5e0b4-5d01-4cb5-9a92-b08214be900f admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:33.776544 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1288/2569] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:33 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:22:33.791481 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ddf5e2-3abc-4c53-a857-02b451883b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:33.792274 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ddf5e2-3abc-4c53-a857-02b451883b3b admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:33.792621 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ddf5e2-3abc-4c53-a857-02b451883b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:33.793132 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ddf5e2-3abc-4c53-a857-02b451883b3b admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:33.793969 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ddf5e2-3abc-4c53-a857-02b451883b3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:33.796505 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ddf5e2-3abc-4c53-a857-02b451883b3b admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:33.821427 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1282/2570] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:22:33 2022] POST /identity/v3/users => generated 287 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:22:33.825778 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c548f375-d384-407b-a9ec-a050049acf07 service cinder] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:33.837480 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c548f375-d384-407b-a9ec-a050049acf07 service cinder] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:33.842486 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ec383-50d4-494d-9860-4d656b8b00aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:33.843153 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ec383-50d4-494d-9860-4d656b8b00aa admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:33.843430 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ec383-50d4-494d-9860-4d656b8b00aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:33.843704 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ec383-50d4-494d-9860-4d656b8b00aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:33.844441 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508ec383-50d4-494d-9860-4d656b8b00aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:33.846086 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508ec383-50d4-494d-9860-4d656b8b00aa admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:33.862585 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1289/2571] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:22:33.866473 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ddf5e2-3abc-4c53-a857-02b451883b3b admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:33.880823 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ddf5e2-3abc-4c53-a857-02b451883b3b admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:33.888769 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72a1e9d-95b8-45b3-8833-6aacc47cbf0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:33.889791 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a1e9d-95b8-45b3-8833-6aacc47cbf0e admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:33.890141 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a1e9d-95b8-45b3-8833-6aacc47cbf0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:33.890480 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a1e9d-95b8-45b3-8833-6aacc47cbf0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:33.891363 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72a1e9d-95b8-45b3-8833-6aacc47cbf0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:33.894282 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72a1e9d-95b8-45b3-8833-6aacc47cbf0e admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:33.902539 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:33.913101 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1283/2572] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:33 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:22:33.917431 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ec383-50d4-494d-9860-4d656b8b00aa admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:33.935995 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ec383-50d4-494d-9860-4d656b8b00aa admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:33.944804 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dd76d3-b6aa-4046-b4d2-9d49be766157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:33.946537 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd76d3-b6aa-4046-b4d2-9d49be766157 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:33.946962 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd76d3-b6aa-4046-b4d2-9d49be766157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:33.947402 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd76d3-b6aa-4046-b4d2-9d49be766157 admin admin] PATH_INFO: `/v3/projects/8d652800501f467f810c467e932c6b8f/users/bde3db230d6249a19a674a55b8d62bfc/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:33.968517 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dd76d3-b6aa-4046-b4d2-9d49be766157 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d652800501f467f810c467e932c6b8f, user_id=bde3db230d6249a19a674a55b8d62bfc, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:33.971252 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dd76d3-b6aa-4046-b4d2-9d49be766157 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:34.002312 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1290/2573] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:22:33 2022] PUT /identity/v3/projects/8d652800501f467f810c467e932c6b8f/users/bde3db230d6249a19a674a55b8d62bfc/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:34.017504 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c88a2-cdb5-4d6d-bebb-4c108c86eb04 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:34.017504 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c88a2-cdb5-4d6d-bebb-4c108c86eb04 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:34.018039 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c88a2-cdb5-4d6d-bebb-4c108c86eb04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:34.056835 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-394c88a2-cdb5-4d6d-bebb-4c108c86eb04 None None] MFA Rules not processed for user `bde3db230d6249a19a674a55b8d62bfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:34.058151 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-394c88a2-cdb5-4d6d-bebb-4c108c86eb04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:34.131189 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1284/2574] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:22:34.142977 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dd76d3-b6aa-4046-b4d2-9d49be766157 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:34.144791 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d2dd76d3-b6aa-4046-b4d2-9d49be766157 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:34.212230 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dd76d3-b6aa-4046-b4d2-9d49be766157 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:34.216092 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282486cf-a6f7-41e9-bd0a-05ebd076b626 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:34.216762 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282486cf-a6f7-41e9-bd0a-05ebd076b626 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:34.217050 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282486cf-a6f7-41e9-bd0a-05ebd076b626 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:34.217323 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282486cf-a6f7-41e9-bd0a-05ebd076b626 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:34.226764 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282486cf-a6f7-41e9-bd0a-05ebd076b626 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:34.228264 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282486cf-a6f7-41e9-bd0a-05ebd076b626 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:34.238064 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1291/2575] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:22:37.276855 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c88a2-cdb5-4d6d-bebb-4c108c86eb04 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:37.344378 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c88a2-cdb5-4d6d-bebb-4c108c86eb04 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:37.348958 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d755d4b3-9ae1-402d-a293-cb3acec63c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:37.349861 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755d4b3-9ae1-402d-a293-cb3acec63c85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:37.350144 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755d4b3-9ae1-402d-a293-cb3acec63c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:37.350426 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755d4b3-9ae1-402d-a293-cb3acec63c85 admin admin] PATH_INFO: `/v3/users/bde3db230d6249a19a674a55b8d62bfc` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:37.351691 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d755d4b3-9ae1-402d-a293-cb3acec63c85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bde3db230d6249a19a674a55b8d62bfc)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:37.353656 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d755d4b3-9ae1-402d-a293-cb3acec63c85 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:37.423131 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-d755d4b3-9ae1-402d-a293-cb3acec63c85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bde3db230d6249a19a674a55b8d62bfc'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:37.432094 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-d755d4b3-9ae1-402d-a293-cb3acec63c85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bde3db230d6249a19a674a55b8d62bfc'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:37.432961 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-d755d4b3-9ae1-402d-a293-cb3acec63c85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bde3db230d6249a19a674a55b8d62bfc'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:37.433704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-d755d4b3-9ae1-402d-a293-cb3acec63c85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bde3db230d6249a19a674a55b8d62bfc'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:37.443676 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-d755d4b3-9ae1-402d-a293-cb3acec63c85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bde3db230d6249a19a674a55b8d62bfc'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:37.467917 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1285/2576] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:22:37 2022] DELETE /identity/v3/users/bde3db230d6249a19a674a55b8d62bfc => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:37.472778 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75634f-628d-474f-a7a5-007544922017 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:37.473332 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75634f-628d-474f-a7a5-007544922017 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:37.473462 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75634f-628d-474f-a7a5-007544922017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:37.498004 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-af75634f-628d-474f-a7a5-007544922017 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:37.500723 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-af75634f-628d-474f-a7a5-007544922017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:37.545800 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:37.586206 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1292/2577] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:37.595948 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d755d4b3-9ae1-402d-a293-cb3acec63c85 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:37.597501 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-d755d4b3-9ae1-402d-a293-cb3acec63c85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:37.609479 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:37.662168 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d755d4b3-9ae1-402d-a293-cb3acec63c85 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:37.666107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5424869e-ebcc-44c8-8b73-ba6421614107 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:37.666799 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424869e-ebcc-44c8-8b73-ba6421614107 service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:37.667086 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424869e-ebcc-44c8-8b73-ba6421614107 service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:37.667359 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424869e-ebcc-44c8-8b73-ba6421614107 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:37.677575 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5424869e-ebcc-44c8-8b73-ba6421614107 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:37.679635 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5424869e-ebcc-44c8-8b73-ba6421614107 service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:37.691368 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1286/2578] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:22:38.053943 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af75634f-628d-474f-a7a5-007544922017 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:38.057787 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-af75634f-628d-474f-a7a5-007544922017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:38.085880 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af75634f-628d-474f-a7a5-007544922017 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:38.092020 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f84006-27f1-4d33-939d-430811b5fa0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:38.093092 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f84006-27f1-4d33-939d-430811b5fa0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:38.093373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f84006-27f1-4d33-939d-430811b5fa0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:38.093684 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f84006-27f1-4d33-939d-430811b5fa0a admin admin] PATH_INFO: `/v3/projects/8d652800501f467f810c467e932c6b8f` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:38.095586 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f84006-27f1-4d33-939d-430811b5fa0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d652800501f467f810c467e932c6b8f)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:38.098900 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f84006-27f1-4d33-939d-430811b5fa0a admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:38.167897 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-45f84006-27f1-4d33-939d-430811b5fa0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d652800501f467f810c467e932c6b8f', 'request_id': 'req-45f84006-27f1-4d33-939d-430811b5fa0a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:38.177855 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-45f84006-27f1-4d33-939d-430811b5fa0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d652800501f467f810c467e932c6b8f', 'request_id': 'req-45f84006-27f1-4d33-939d-430811b5fa0a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:38.189079 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-45f84006-27f1-4d33-939d-430811b5fa0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d652800501f467f810c467e932c6b8f', 'request_id': 'req-45f84006-27f1-4d33-939d-430811b5fa0a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:38.205399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-45f84006-27f1-4d33-939d-430811b5fa0a admin admin] The token cache is being invalidate because project 8d652800501f467f810c467e932c6b8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:22:38.205788 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-45f84006-27f1-4d33-939d-430811b5fa0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:38.207627 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-45f84006-27f1-4d33-939d-430811b5fa0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:38.209799 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1293/2579] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:22:38 2022] DELETE /identity/v3/projects/8d652800501f467f810c467e932c6b8f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:38.215789 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139df85d-a757-49e9-8f50-908852307dc3 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:38.216214 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139df85d-a757-49e9-8f50-908852307dc3 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:38.216495 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139df85d-a757-49e9-8f50-908852307dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:38.238140 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-139df85d-a757-49e9-8f50-908852307dc3 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:38.239377 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-139df85d-a757-49e9-8f50-908852307dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:38.307173 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1287/2580] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:38.314915 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45f56f-5498-44f1-802b-4302f7d96edf None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:38.315192 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45f56f-5498-44f1-802b-4302f7d96edf None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:38.315464 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45f56f-5498-44f1-802b-4302f7d96edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:38.336784 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-9d45f56f-5498-44f1-802b-4302f7d96edf None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:38.338065 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-9d45f56f-5498-44f1-802b-4302f7d96edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:38.347765 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1294/2581] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:38.352449 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139df85d-a757-49e9-8f50-908852307dc3 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:38.363311 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139df85d-a757-49e9-8f50-908852307dc3 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:38.368964 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10774e63-b28e-4e4e-beb0-7b116824de64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:38.369657 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10774e63-b28e-4e4e-beb0-7b116824de64 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:38.369939 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10774e63-b28e-4e4e-beb0-7b116824de64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:38.370211 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10774e63-b28e-4e4e-beb0-7b116824de64 admin admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:38.370961 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10774e63-b28e-4e4e-beb0-7b116824de64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:22:38.371272 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10774e63-b28e-4e4e-beb0-7b116824de64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:38.372893 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10774e63-b28e-4e4e-beb0-7b116824de64 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:38.386207 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1288/2582] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:22:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:22:38.390849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d45f56f-5498-44f1-802b-4302f7d96edf None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:38.401111 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d45f56f-5498-44f1-802b-4302f7d96edf None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:38.406091 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2567ef-f074-4014-8313-2f28d209a0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:38.406789 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2567ef-f074-4014-8313-2f28d209a0f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:38.407067 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2567ef-f074-4014-8313-2f28d209a0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:38.407349 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2567ef-f074-4014-8313-2f28d209a0f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:38.408080 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2567ef-f074-4014-8313-2f28d209a0f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:38.410063 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2567ef-f074-4014-8313-2f28d209a0f1 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:38.439678 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1295/2583] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:22:38 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:22:38.443878 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10774e63-b28e-4e4e-beb0-7b116824de64 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:38.454776 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10774e63-b28e-4e4e-beb0-7b116824de64 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:38.517310 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a30d1c-a055-4c2d-96d7-93e388ce854e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:38.517997 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a30d1c-a055-4c2d-96d7-93e388ce854e admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:38.518276 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a30d1c-a055-4c2d-96d7-93e388ce854e admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:38.518857 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a30d1c-a055-4c2d-96d7-93e388ce854e admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:38.519593 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a30d1c-a055-4c2d-96d7-93e388ce854e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:38.521582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a30d1c-a055-4c2d-96d7-93e388ce854e admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:38.553761 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1289/2584] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:22:38 2022] POST /identity/v3/users => generated 284 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:22:38.554971 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2567ef-f074-4014-8313-2f28d209a0f1 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:38.566198 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2567ef-f074-4014-8313-2f28d209a0f1 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:38.572185 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f5bb8-62cd-4036-b093-fba15c904f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:38.572893 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f5bb8-62cd-4036-b093-fba15c904f21 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:38.573214 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f5bb8-62cd-4036-b093-fba15c904f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:38.573517 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f5bb8-62cd-4036-b093-fba15c904f21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:38.574355 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773f5bb8-62cd-4036-b093-fba15c904f21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:38.575998 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773f5bb8-62cd-4036-b093-fba15c904f21 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:38.588263 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1296/2585] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:22:38.592808 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a30d1c-a055-4c2d-96d7-93e388ce854e admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:38.605725 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a30d1c-a055-4c2d-96d7-93e388ce854e admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:38.611719 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d482a79-7adc-4694-a670-eb326b3fb8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:38.612408 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d482a79-7adc-4694-a670-eb326b3fb8cf admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:38.612689 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d482a79-7adc-4694-a670-eb326b3fb8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:38.612992 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d482a79-7adc-4694-a670-eb326b3fb8cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:38.613738 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d482a79-7adc-4694-a670-eb326b3fb8cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:38.615342 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d482a79-7adc-4694-a670-eb326b3fb8cf admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:38.628578 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1290/2586] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:22:38.632534 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f5bb8-62cd-4036-b093-fba15c904f21 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:38.646961 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f5bb8-62cd-4036-b093-fba15c904f21 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:38.653392 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea40da16-c2f2-407c-8afb-8091d78119b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:38.654491 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea40da16-c2f2-407c-8afb-8091d78119b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:38.654860 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea40da16-c2f2-407c-8afb-8091d78119b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:38.655205 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea40da16-c2f2-407c-8afb-8091d78119b5 admin admin] PATH_INFO: `/v3/projects/1a1c6bca01a94d56a5e8733239eda8aa/users/0c9f24f0f2ba401ba4cfd4bb6787c888/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:38.682500 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea40da16-c2f2-407c-8afb-8091d78119b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a1c6bca01a94d56a5e8733239eda8aa, user_id=0c9f24f0f2ba401ba4cfd4bb6787c888, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:38.686506 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea40da16-c2f2-407c-8afb-8091d78119b5 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:38.707698 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1297/2587] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:22:38 2022] PUT /identity/v3/projects/1a1c6bca01a94d56a5e8733239eda8aa/users/0c9f24f0f2ba401ba4cfd4bb6787c888/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:22:38.719713 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b63c0d-5c3f-40d6-aaf3-a2da1c2e370f None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:38.720152 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b63c0d-5c3f-40d6-aaf3-a2da1c2e370f None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:38.720359 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b63c0d-5c3f-40d6-aaf3-a2da1c2e370f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:38.746910 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea40da16-c2f2-407c-8afb-8091d78119b5 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:38.749576 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ea40da16-c2f2-407c-8afb-8091d78119b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:38.750509 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-87b63c0d-5c3f-40d6-aaf3-a2da1c2e370f None None] MFA Rules not processed for user `0c9f24f0f2ba401ba4cfd4bb6787c888`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:38.751722 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-87b63c0d-5c3f-40d6-aaf3-a2da1c2e370f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:38.757750 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:38.843438 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:38.848349 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea40da16-c2f2-407c-8afb-8091d78119b5 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:38.851051 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1291/2588] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:38 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:38.856870 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcb21cb-59a6-428e-9ed9-4044310cc516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:38.860489 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb21cb-59a6-428e-9ed9-4044310cc516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:38.860489 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb21cb-59a6-428e-9ed9-4044310cc516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:38.860904 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb21cb-59a6-428e-9ed9-4044310cc516 admin admin] PATH_INFO: `/v3/users/60794b2c0e734b7b99cd66d72087a6c6` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:38.862952 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcb21cb-59a6-428e-9ed9-4044310cc516 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60794b2c0e734b7b99cd66d72087a6c6)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:38.865210 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcb21cb-59a6-428e-9ed9-4044310cc516 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:38.870154 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b63c0d-5c3f-40d6-aaf3-a2da1c2e370f None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:38.872078 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-87b63c0d-5c3f-40d6-aaf3-a2da1c2e370f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:38.921569 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-afcb21cb-59a6-428e-9ed9-4044310cc516 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60794b2c0e734b7b99cd66d72087a6c6'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:38.929636 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-afcb21cb-59a6-428e-9ed9-4044310cc516 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60794b2c0e734b7b99cd66d72087a6c6'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:38.930397 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-afcb21cb-59a6-428e-9ed9-4044310cc516 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60794b2c0e734b7b99cd66d72087a6c6'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:38.931095 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-afcb21cb-59a6-428e-9ed9-4044310cc516 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60794b2c0e734b7b99cd66d72087a6c6'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:38.946038 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-afcb21cb-59a6-428e-9ed9-4044310cc516 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60794b2c0e734b7b99cd66d72087a6c6'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:38.981181 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1298/2589] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:22:38 2022] DELETE /identity/v3/users/60794b2c0e734b7b99cd66d72087a6c6 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:38.983037 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee669866-4471-4f4d-9cfe-b5f315ce0415 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:38.990548 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee669866-4471-4f4d-9cfe-b5f315ce0415 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:38.990548 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee669866-4471-4f4d-9cfe-b5f315ce0415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:39.000686 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b63c0d-5c3f-40d6-aaf3-a2da1c2e370f None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:39.021139 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-ee669866-4471-4f4d-9cfe-b5f315ce0415 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:39.023075 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ee669866-4471-4f4d-9cfe-b5f315ce0415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:39.073514 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e308089-24a4-4a26-9505-f010be4a27a8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26cf962d151b4da7a82c33ef23b342e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:39.074100 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e308089-24a4-4a26-9505-f010be4a27a8 service glance] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:39.074381 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e308089-24a4-4a26-9505-f010be4a27a8 service glance] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:39.074872 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e308089-24a4-4a26-9505-f010be4a27a8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:39.076873 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-5e308089-24a4-4a26-9505-f010be4a27a8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:39.122223 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1299/2590] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:39.137648 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee669866-4471-4f4d-9cfe-b5f315ce0415 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:39.139617 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ee669866-4471-4f4d-9cfe-b5f315ce0415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:39.160812 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e308089-24a4-4a26-9505-f010be4a27a8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:39.162449 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e308089-24a4-4a26-9505-f010be4a27a8 service glance] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:39.168151 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:39.187847 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1292/2591] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:38 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:22:39.241584 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee669866-4471-4f4d-9cfe-b5f315ce0415 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:39.249640 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184001d2-01eb-4735-b91f-7666dd03b4dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:39.250367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184001d2-01eb-4735-b91f-7666dd03b4dc service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:39.252542 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184001d2-01eb-4735-b91f-7666dd03b4dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:39.252911 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184001d2-01eb-4735-b91f-7666dd03b4dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:39.271411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184001d2-01eb-4735-b91f-7666dd03b4dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:39.273144 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184001d2-01eb-4735-b91f-7666dd03b4dc service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:39.286681 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1300/2592] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:22:39.823779 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e308089-24a4-4a26-9505-f010be4a27a8 service glance] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:39.827214 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-5e308089-24a4-4a26-9505-f010be4a27a8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:39.854155 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e308089-24a4-4a26-9505-f010be4a27a8 service glance] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:39.863487 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d328e3-c351-49b4-8825-2cdfdd5d0e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:39.865632 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d328e3-c351-49b4-8825-2cdfdd5d0e42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:39.866290 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d328e3-c351-49b4-8825-2cdfdd5d0e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:39.867676 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d328e3-c351-49b4-8825-2cdfdd5d0e42 admin admin] PATH_INFO: `/v3/projects/b609118f5342413b8ec541b95dbe8d27` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:39.871093 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d328e3-c351-49b4-8825-2cdfdd5d0e42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b609118f5342413b8ec541b95dbe8d27)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:39.872983 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d328e3-c351-49b4-8825-2cdfdd5d0e42 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:39.942158 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-87d328e3-c351-49b4-8825-2cdfdd5d0e42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b609118f5342413b8ec541b95dbe8d27', 'request_id': 'req-87d328e3-c351-49b4-8825-2cdfdd5d0e42'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:39.951094 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-87d328e3-c351-49b4-8825-2cdfdd5d0e42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b609118f5342413b8ec541b95dbe8d27', 'request_id': 'req-87d328e3-c351-49b4-8825-2cdfdd5d0e42'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:39.960246 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-87d328e3-c351-49b4-8825-2cdfdd5d0e42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b609118f5342413b8ec541b95dbe8d27', 'request_id': 'req-87d328e3-c351-49b4-8825-2cdfdd5d0e42'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:39.977668 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-87d328e3-c351-49b4-8825-2cdfdd5d0e42 admin admin] The token cache is being invalidate because project b609118f5342413b8ec541b95dbe8d27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:22:39.977946 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-87d328e3-c351-49b4-8825-2cdfdd5d0e42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:39.978467 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-87d328e3-c351-49b4-8825-2cdfdd5d0e42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:39.980097 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1293/2593] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:22:39 2022] DELETE /identity/v3/projects/b609118f5342413b8ec541b95dbe8d27 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:39.984063 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184001d2-01eb-4735-b91f-7666dd03b4dc service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:39.994173 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184001d2-01eb-4735-b91f-7666dd03b4dc service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:40.014380 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:40.058592 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63bda9c-4cd4-4765-9f7d-b501b0b971c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:40.059692 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63bda9c-4cd4-4765-9f7d-b501b0b971c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:40.060155 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63bda9c-4cd4-4765-9f7d-b501b0b971c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:40.060440 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63bda9c-4cd4-4765-9f7d-b501b0b971c1 admin admin] PATH_INFO: `/v3/users/d2fb6e6f971247afb7ea0972f2c8d914` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:40.061974 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63bda9c-4cd4-4765-9f7d-b501b0b971c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2fb6e6f971247afb7ea0972f2c8d914)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:40.064580 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63bda9c-4cd4-4765-9f7d-b501b0b971c1 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:40.119240 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-e63bda9c-4cd4-4765-9f7d-b501b0b971c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2fb6e6f971247afb7ea0972f2c8d914'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:40.129229 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-e63bda9c-4cd4-4765-9f7d-b501b0b971c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2fb6e6f971247afb7ea0972f2c8d914'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:40.129977 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-e63bda9c-4cd4-4765-9f7d-b501b0b971c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2fb6e6f971247afb7ea0972f2c8d914'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:40.130568 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-e63bda9c-4cd4-4765-9f7d-b501b0b971c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2fb6e6f971247afb7ea0972f2c8d914'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:40.140371 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-e63bda9c-4cd4-4765-9f7d-b501b0b971c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2fb6e6f971247afb7ea0972f2c8d914'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:40.159364 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1301/2594] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:22:39 2022] DELETE /identity/v3/users/d2fb6e6f971247afb7ea0972f2c8d914 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:40.458561 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d328e3-c351-49b4-8825-2cdfdd5d0e42 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:40.460406 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-87d328e3-c351-49b4-8825-2cdfdd5d0e42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:40.475078 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d328e3-c351-49b4-8825-2cdfdd5d0e42 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:40.540189 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f914d-ecbd-4ff6-85cb-94630c104160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:40.541104 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f914d-ecbd-4ff6-85cb-94630c104160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:40.541383 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f914d-ecbd-4ff6-85cb-94630c104160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:40.541672 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f914d-ecbd-4ff6-85cb-94630c104160 admin admin] PATH_INFO: `/v3/projects/cf650ec2097548e0a50e2d68f58cb777` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:40.542829 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f914d-ecbd-4ff6-85cb-94630c104160 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf650ec2097548e0a50e2d68f58cb777)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:40.544754 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f914d-ecbd-4ff6-85cb-94630c104160 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:40.598001 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-535f914d-ecbd-4ff6-85cb-94630c104160 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf650ec2097548e0a50e2d68f58cb777', 'request_id': 'req-535f914d-ecbd-4ff6-85cb-94630c104160'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:40.604538 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-535f914d-ecbd-4ff6-85cb-94630c104160 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf650ec2097548e0a50e2d68f58cb777', 'request_id': 'req-535f914d-ecbd-4ff6-85cb-94630c104160'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:40.613116 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-535f914d-ecbd-4ff6-85cb-94630c104160 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf650ec2097548e0a50e2d68f58cb777', 'request_id': 'req-535f914d-ecbd-4ff6-85cb-94630c104160'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:40.624248 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-535f914d-ecbd-4ff6-85cb-94630c104160 admin admin] The token cache is being invalidate because project cf650ec2097548e0a50e2d68f58cb777 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:22:40.624465 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-535f914d-ecbd-4ff6-85cb-94630c104160 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:40.625233 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-535f914d-ecbd-4ff6-85cb-94630c104160 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:40.627011 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1294/2595] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:22:40 2022] DELETE /identity/v3/projects/cf650ec2097548e0a50e2d68f58cb777 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:22:40.632117 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf5d4bc-a2e8-4958-afd0-e442dd1b7cf9 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:40.632468 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf5d4bc-a2e8-4958-afd0-e442dd1b7cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:40.633222 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf5d4bc-a2e8-4958-afd0-e442dd1b7cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:40.651322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-4cf5d4bc-a2e8-4958-afd0-e442dd1b7cf9 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:40.652689 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-4cf5d4bc-a2e8-4958-afd0-e442dd1b7cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:40.721229 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1302/2596] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:22:40.730177 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55662e5c-49db-47c5-8c84-847663e3d657 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:40.730740 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55662e5c-49db-47c5-8c84-847663e3d657 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:40.731110 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55662e5c-49db-47c5-8c84-847663e3d657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:40.754109 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-55662e5c-49db-47c5-8c84-847663e3d657 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:40.755710 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-55662e5c-49db-47c5-8c84-847663e3d657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:40.765221 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1295/2597] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:40.771423 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf5d4bc-a2e8-4958-afd0-e442dd1b7cf9 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:40.782372 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf5d4bc-a2e8-4958-afd0-e442dd1b7cf9 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:40.787081 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0493607a-6907-44dd-8a4d-c59ce26758f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:40.787736 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0493607a-6907-44dd-8a4d-c59ce26758f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:40.788013 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0493607a-6907-44dd-8a4d-c59ce26758f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:40.788252 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0493607a-6907-44dd-8a4d-c59ce26758f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:40.789598 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0493607a-6907-44dd-8a4d-c59ce26758f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:22:40.789901 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0493607a-6907-44dd-8a4d-c59ce26758f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:40.791476 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0493607a-6907-44dd-8a4d-c59ce26758f7 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:40.803568 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1303/2598] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:22:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:22:40.808364 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55662e5c-49db-47c5-8c84-847663e3d657 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:40.814436 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:40.818393 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55662e5c-49db-47c5-8c84-847663e3d657 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:40.825511 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b005b8-f5cc-46f2-86b6-8305ed4cc93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:40.826249 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b005b8-f5cc-46f2-86b6-8305ed4cc93d admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:40.826857 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b005b8-f5cc-46f2-86b6-8305ed4cc93d admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:40.827192 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b005b8-f5cc-46f2-86b6-8305ed4cc93d admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:40.828000 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b005b8-f5cc-46f2-86b6-8305ed4cc93d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:40.830157 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b005b8-f5cc-46f2-86b6-8305ed4cc93d admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:40.863044 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1296/2599] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:22:40 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:22:40.867441 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0493607a-6907-44dd-8a4d-c59ce26758f7 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:40.878463 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0493607a-6907-44dd-8a4d-c59ce26758f7 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:40.904636 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:40.947640 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ddadda-fa76-4775-955d-02489914a5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:40.948332 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ddadda-fa76-4775-955d-02489914a5f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:40.948616 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ddadda-fa76-4775-955d-02489914a5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:40.948883 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ddadda-fa76-4775-955d-02489914a5f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:40.949708 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ddadda-fa76-4775-955d-02489914a5f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:40.951560 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ddadda-fa76-4775-955d-02489914a5f9 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:40.974242 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1304/2600] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:22:40 2022] POST /identity/v3/users => generated 287 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:22:40.978649 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b005b8-f5cc-46f2-86b6-8305ed4cc93d admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:40.989753 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b005b8-f5cc-46f2-86b6-8305ed4cc93d admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:40.995604 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a06a92-a376-4fd1-b3b9-bd077a7c866c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:40.996300 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a06a92-a376-4fd1-b3b9-bd077a7c866c admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:40.996581 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a06a92-a376-4fd1-b3b9-bd077a7c866c admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:40.996882 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a06a92-a376-4fd1-b3b9-bd077a7c866c admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:40.997789 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a06a92-a376-4fd1-b3b9-bd077a7c866c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:40.999433 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a06a92-a376-4fd1-b3b9-bd077a7c866c admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:41.010782 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1297/2601] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:22:41.014929 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ddadda-fa76-4775-955d-02489914a5f9 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:41.026350 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ddadda-fa76-4775-955d-02489914a5f9 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:41.033636 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51877b4c-89a4-4f5d-98ac-46f4717881ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:41.034666 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51877b4c-89a4-4f5d-98ac-46f4717881ab admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:41.035089 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51877b4c-89a4-4f5d-98ac-46f4717881ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:41.035542 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51877b4c-89a4-4f5d-98ac-46f4717881ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:41.036379 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51877b4c-89a4-4f5d-98ac-46f4717881ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:41.038464 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51877b4c-89a4-4f5d-98ac-46f4717881ab admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:41.054502 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1305/2602] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:41 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:22:41.058361 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a06a92-a376-4fd1-b3b9-bd077a7c866c admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:41.071904 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a06a92-a376-4fd1-b3b9-bd077a7c866c admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:41.077825 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459d1fb-dfd9-43d6-ae50-c6fec1f0391e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:41.078891 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459d1fb-dfd9-43d6-ae50-c6fec1f0391e admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:41.079243 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459d1fb-dfd9-43d6-ae50-c6fec1f0391e admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:41.081907 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459d1fb-dfd9-43d6-ae50-c6fec1f0391e admin admin] PATH_INFO: `/v3/projects/befdb890aa684628ac4bd08947f784f2/users/f3ff4481fc5248a8b2f158ec4b69fda9/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:41.107552 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2459d1fb-dfd9-43d6-ae50-c6fec1f0391e admin admin] RBAC: Authorizing `identity:create_grant(project_id=befdb890aa684628ac4bd08947f784f2, user_id=f3ff4481fc5248a8b2f158ec4b69fda9, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:41.110746 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2459d1fb-dfd9-43d6-ae50-c6fec1f0391e admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:41.136144 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1298/2603] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:22:41 2022] PUT /identity/v3/projects/befdb890aa684628ac4bd08947f784f2/users/f3ff4481fc5248a8b2f158ec4b69fda9/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:41.146683 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83495f-48df-47f1-b11c-9e5cc881ecc0 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:41.147189 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83495f-48df-47f1-b11c-9e5cc881ecc0 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:41.147305 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83495f-48df-47f1-b11c-9e5cc881ecc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:41.180284 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-ea83495f-48df-47f1-b11c-9e5cc881ecc0 None None] MFA Rules not processed for user `f3ff4481fc5248a8b2f158ec4b69fda9`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:41.181560 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ea83495f-48df-47f1-b11c-9e5cc881ecc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:41.266887 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1306/2604] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:41.279213 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459d1fb-dfd9-43d6-ae50-c6fec1f0391e admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:41.281349 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-2459d1fb-dfd9-43d6-ae50-c6fec1f0391e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:41.349984 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459d1fb-dfd9-43d6-ae50-c6fec1f0391e admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:41.354645 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3473e98-5664-4d58-84de-9683bd89a579 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:41.355325 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3473e98-5664-4d58-84de-9683bd89a579 service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:41.355607 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3473e98-5664-4d58-84de-9683bd89a579 service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:41.355881 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3473e98-5664-4d58-84de-9683bd89a579 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:41.365744 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3473e98-5664-4d58-84de-9683bd89a579 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:41.367189 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3473e98-5664-4d58-84de-9683bd89a579 service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:41.377731 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1299/2605] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:41 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:22:41.397848 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea83495f-48df-47f1-b11c-9e5cc881ecc0 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:41.468366 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea83495f-48df-47f1-b11c-9e5cc881ecc0 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:41.473291 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a880f448-bc92-4db2-b2fb-1159d5f9b2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:41.474183 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a880f448-bc92-4db2-b2fb-1159d5f9b2d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:41.474456 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a880f448-bc92-4db2-b2fb-1159d5f9b2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:41.474821 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a880f448-bc92-4db2-b2fb-1159d5f9b2d2 admin admin] PATH_INFO: `/v3/users/f3ff4481fc5248a8b2f158ec4b69fda9` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:41.476980 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a880f448-bc92-4db2-b2fb-1159d5f9b2d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3ff4481fc5248a8b2f158ec4b69fda9)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:41.484960 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a880f448-bc92-4db2-b2fb-1159d5f9b2d2 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:41.526598 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:41.536207 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a880f448-bc92-4db2-b2fb-1159d5f9b2d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3ff4481fc5248a8b2f158ec4b69fda9'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:41.544347 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a880f448-bc92-4db2-b2fb-1159d5f9b2d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3ff4481fc5248a8b2f158ec4b69fda9'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:41.545102 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a880f448-bc92-4db2-b2fb-1159d5f9b2d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3ff4481fc5248a8b2f158ec4b69fda9'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:41.545695 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a880f448-bc92-4db2-b2fb-1159d5f9b2d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3ff4481fc5248a8b2f158ec4b69fda9'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:41.554684 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-a880f448-bc92-4db2-b2fb-1159d5f9b2d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3ff4481fc5248a8b2f158ec4b69fda9'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:41.579636 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1307/2606] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:22:41 2022] DELETE /identity/v3/users/f3ff4481fc5248a8b2f158ec4b69fda9 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:41.584913 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede40df3-7b67-4a72-96e8-88272c1ed9e4 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:41.585393 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede40df3-7b67-4a72-96e8-88272c1ed9e4 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:41.585742 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede40df3-7b67-4a72-96e8-88272c1ed9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:41.603903 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-ede40df3-7b67-4a72-96e8-88272c1ed9e4 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:41.605282 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-ede40df3-7b67-4a72-96e8-88272c1ed9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:41.685225 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1300/2607] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:41.696884 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a880f448-bc92-4db2-b2fb-1159d5f9b2d2 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:41.698563 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a880f448-bc92-4db2-b2fb-1159d5f9b2d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:41.773195 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a880f448-bc92-4db2-b2fb-1159d5f9b2d2 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:41.779209 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7548c9e1-ba58-4de1-a143-0f8b7b269dec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:41.780543 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7548c9e1-ba58-4de1-a143-0f8b7b269dec service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:41.781614 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7548c9e1-ba58-4de1-a143-0f8b7b269dec service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:41.782053 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7548c9e1-ba58-4de1-a143-0f8b7b269dec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:41.793815 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7548c9e1-ba58-4de1-a143-0f8b7b269dec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:41.795347 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7548c9e1-ba58-4de1-a143-0f8b7b269dec service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:41.807202 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1308/2608] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:22:42.123114 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede40df3-7b67-4a72-96e8-88272c1ed9e4 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:42.124856 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-ede40df3-7b67-4a72-96e8-88272c1ed9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:42.142889 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede40df3-7b67-4a72-96e8-88272c1ed9e4 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:42.149823 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4457a10a-445c-448a-ba8e-6aabc219856a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:42.150870 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457a10a-445c-448a-ba8e-6aabc219856a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:42.151162 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457a10a-445c-448a-ba8e-6aabc219856a admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:42.151438 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457a10a-445c-448a-ba8e-6aabc219856a admin admin] PATH_INFO: `/v3/projects/befdb890aa684628ac4bd08947f784f2` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:42.152646 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4457a10a-445c-448a-ba8e-6aabc219856a admin admin] RBAC: Authorizing `identity:delete_project(project_id=befdb890aa684628ac4bd08947f784f2)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:42.154660 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4457a10a-445c-448a-ba8e-6aabc219856a admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:42.210106 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:42.213827 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-4457a10a-445c-448a-ba8e-6aabc219856a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'befdb890aa684628ac4bd08947f784f2', 'request_id': 'req-4457a10a-445c-448a-ba8e-6aabc219856a'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:42.222620 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-4457a10a-445c-448a-ba8e-6aabc219856a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'befdb890aa684628ac4bd08947f784f2', 'request_id': 'req-4457a10a-445c-448a-ba8e-6aabc219856a'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:42.232179 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-4457a10a-445c-448a-ba8e-6aabc219856a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'befdb890aa684628ac4bd08947f784f2', 'request_id': 'req-4457a10a-445c-448a-ba8e-6aabc219856a'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:42.245762 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-4457a10a-445c-448a-ba8e-6aabc219856a admin admin] The token cache is being invalidate because project befdb890aa684628ac4bd08947f784f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:22:42.246159 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-4457a10a-445c-448a-ba8e-6aabc219856a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:42.247028 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-4457a10a-445c-448a-ba8e-6aabc219856a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:42.248954 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1301/2609] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:22:42 2022] DELETE /identity/v3/projects/befdb890aa684628ac4bd08947f784f2 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:42.254354 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a48d81-c9c2-4584-b919-5279e6c895e3 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:42.254876 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a48d81-c9c2-4584-b919-5279e6c895e3 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:42.254876 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a48d81-c9c2-4584-b919-5279e6c895e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:42.277025 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-27a48d81-c9c2-4584-b919-5279e6c895e3 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:42.278255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-27a48d81-c9c2-4584-b919-5279e6c895e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:42.357049 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1309/2610] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:42.366120 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d9079-e6d6-4966-b708-54622e696312 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:42.366282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a48d81-c9c2-4584-b919-5279e6c895e3 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:42.366533 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d9079-e6d6-4966-b708-54622e696312 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:42.366886 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d9079-e6d6-4966-b708-54622e696312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:42.368366 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-27a48d81-c9c2-4584-b919-5279e6c895e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:42.386151 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-8d8d9079-e6d6-4966-b708-54622e696312 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:42.387424 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-8d8d9079-e6d6-4966-b708-54622e696312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:42.396164 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1302/2611] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:42.400712 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8d9079-e6d6-4966-b708-54622e696312 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:42.414970 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8d9079-e6d6-4966-b708-54622e696312 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:42.423118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20f87f0-9be1-478e-8678-129e0cc84a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:42.424187 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f87f0-9be1-478e-8678-129e0cc84a94 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:42.424607 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f87f0-9be1-478e-8678-129e0cc84a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:42.425095 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20f87f0-9be1-478e-8678-129e0cc84a94 admin admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:42.426280 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20f87f0-9be1-478e-8678-129e0cc84a94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:22:42.426676 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20f87f0-9be1-478e-8678-129e0cc84a94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:42.429378 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20f87f0-9be1-478e-8678-129e0cc84a94 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:42.439443 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a48d81-c9c2-4584-b919-5279e6c895e3 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:42.448567 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1303/2612] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:22:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:22:42.453091 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20f87f0-9be1-478e-8678-129e0cc84a94 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:42.455663 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4672c59d-7ced-491a-abc7-c72bf9e8cc53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:42.456343 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672c59d-7ced-491a-abc7-c72bf9e8cc53 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:42.456622 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672c59d-7ced-491a-abc7-c72bf9e8cc53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:42.456874 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4672c59d-7ced-491a-abc7-c72bf9e8cc53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:42.459137 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-4672c59d-7ced-491a-abc7-c72bf9e8cc53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:42.464844 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20f87f0-9be1-478e-8678-129e0cc84a94 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:42.469703 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6919ef-75ca-434b-842a-7253c55b79b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:42.470366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6919ef-75ca-434b-842a-7253c55b79b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:42.470672 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6919ef-75ca-434b-842a-7253c55b79b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:42.470942 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6919ef-75ca-434b-842a-7253c55b79b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:42.471632 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6919ef-75ca-434b-842a-7253c55b79b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:42.473575 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6919ef-75ca-434b-842a-7253c55b79b0 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:42.473690 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4672c59d-7ced-491a-abc7-c72bf9e8cc53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:42.475033 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4672c59d-7ced-491a-abc7-c72bf9e8cc53 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:42.480847 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:42.486985 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1310/2613] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:22:42.506440 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1304/2614] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:22:42 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:22:42.513740 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4672c59d-7ced-491a-abc7-c72bf9e8cc53 service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:42.532259 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4672c59d-7ced-491a-abc7-c72bf9e8cc53 service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:42.596601 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84054ddc-5385-4aae-91c6-402c8c3707be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:42.597498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84054ddc-5385-4aae-91c6-402c8c3707be admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:42.597891 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84054ddc-5385-4aae-91c6-402c8c3707be admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:42.598447 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84054ddc-5385-4aae-91c6-402c8c3707be admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:42.599347 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84054ddc-5385-4aae-91c6-402c8c3707be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:42.602024 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84054ddc-5385-4aae-91c6-402c8c3707be admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:42.630872 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1311/2615] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:22:42 2022] POST /identity/v3/users => generated 287 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:22:42.635694 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6919ef-75ca-434b-842a-7253c55b79b0 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:42.646896 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6919ef-75ca-434b-842a-7253c55b79b0 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:42.652597 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332c1d7-ef50-4b20-9641-f7c123842c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:42.653343 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332c1d7-ef50-4b20-9641-f7c123842c7a admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:42.653624 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332c1d7-ef50-4b20-9641-f7c123842c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:42.653895 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332c1d7-ef50-4b20-9641-f7c123842c7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:42.654642 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9332c1d7-ef50-4b20-9641-f7c123842c7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:42.656261 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9332c1d7-ef50-4b20-9641-f7c123842c7a admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:42.668360 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1305/2616] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:22:42.672783 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84054ddc-5385-4aae-91c6-402c8c3707be admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:42.685410 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84054ddc-5385-4aae-91c6-402c8c3707be admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:42.690181 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1ef39-6a91-4666-882c-ae036ed1d093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:42.690882 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1ef39-6a91-4666-882c-ae036ed1d093 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:42.691162 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1ef39-6a91-4666-882c-ae036ed1d093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:42.691470 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1ef39-6a91-4666-882c-ae036ed1d093 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:42.692206 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d1ef39-6a91-4666-882c-ae036ed1d093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:42.694080 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d1ef39-6a91-4666-882c-ae036ed1d093 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:42.705155 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1312/2617] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:22:42.709646 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332c1d7-ef50-4b20-9641-f7c123842c7a admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:42.720691 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332c1d7-ef50-4b20-9641-f7c123842c7a admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:42.725864 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badf3517-1980-4f4c-b409-25dee571c901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:42.727635 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf3517-1980-4f4c-b409-25dee571c901 admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:42.727901 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf3517-1980-4f4c-b409-25dee571c901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:42.728253 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf3517-1980-4f4c-b409-25dee571c901 admin admin] PATH_INFO: `/v3/projects/8b70340671074b2aa4c80384cb6f1c85/users/e0e171895ccc4f1cbc69ead21175fc5d/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:42.749021 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badf3517-1980-4f4c-b409-25dee571c901 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b70340671074b2aa4c80384cb6f1c85, user_id=e0e171895ccc4f1cbc69ead21175fc5d, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:42.750831 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badf3517-1980-4f4c-b409-25dee571c901 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:42.768402 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1306/2618] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:22:42 2022] PUT /identity/v3/projects/8b70340671074b2aa4c80384cb6f1c85/users/e0e171895ccc4f1cbc69ead21175fc5d/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:22:42.773895 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b52261-0a06-4304-97a2-3ca7cc83b588 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:42.774172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b52261-0a06-4304-97a2-3ca7cc83b588 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:42.774449 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b52261-0a06-4304-97a2-3ca7cc83b588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:42.794232 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-e1b52261-0a06-4304-97a2-3ca7cc83b588 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:42.795819 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-e1b52261-0a06-4304-97a2-3ca7cc83b588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:42.871995 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1313/2619] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:42.886424 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badf3517-1980-4f4c-b409-25dee571c901 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:42.953244 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badf3517-1980-4f4c-b409-25dee571c901 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:42.962120 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85108102-91a6-47d4-817d-6efd92a22766 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:42.962801 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85108102-91a6-47d4-817d-6efd92a22766 service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:42.963080 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85108102-91a6-47d4-817d-6efd92a22766 service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:42.963344 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85108102-91a6-47d4-817d-6efd92a22766 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:42.973332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85108102-91a6-47d4-817d-6efd92a22766 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:42.974681 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85108102-91a6-47d4-817d-6efd92a22766 service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:42.985818 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1307/2620] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:22:43.968986 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27988f9-3475-441f-9d53-6dcc17d6fea0 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:43.969935 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27988f9-3475-441f-9d53-6dcc17d6fea0 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:43.970388 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27988f9-3475-441f-9d53-6dcc17d6fea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:44.009062 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-b27988f9-3475-441f-9d53-6dcc17d6fea0 None None] MFA Rules not processed for user `e0e171895ccc4f1cbc69ead21175fc5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:44.010303 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-b27988f9-3475-441f-9d53-6dcc17d6fea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:44.095111 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1314/2621] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:44.107721 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85108102-91a6-47d4-817d-6efd92a22766 service neutron] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:44.111046 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-85108102-91a6-47d4-817d-6efd92a22766 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:44.193633 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85108102-91a6-47d4-817d-6efd92a22766 service neutron] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:44.203887 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd87c516-f038-40c5-a693-6f97414d29e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:44.204566 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87c516-f038-40c5-a693-6f97414d29e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:44.204851 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87c516-f038-40c5-a693-6f97414d29e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:44.205186 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87c516-f038-40c5-a693-6f97414d29e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:44.215259 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd87c516-f038-40c5-a693-6f97414d29e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:44.216702 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd87c516-f038-40c5-a693-6f97414d29e8 service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:44.226835 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1308/2622] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:22:45.659660 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a701e-40eb-4f8d-985b-064ef9e17cc4 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:45.660569 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a701e-40eb-4f8d-985b-064ef9e17cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:45.661083 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a701e-40eb-4f8d-985b-064ef9e17cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:45.681815 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-786a701e-40eb-4f8d-985b-064ef9e17cc4 None None] MFA Rules not processed for user `e0e171895ccc4f1cbc69ead21175fc5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:45.683472 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-786a701e-40eb-4f8d-985b-064ef9e17cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:45.700311 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1315/2623] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:46.039599 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd87c516-f038-40c5-a693-6f97414d29e8 service cinder] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:46.041135 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-dd87c516-f038-40c5-a693-6f97414d29e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:46.049629 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:46.056896 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd87c516-f038-40c5-a693-6f97414d29e8 service cinder] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:46.068404 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f2833-36a3-4cee-bc7e-93faf5c75c97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:46.069290 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f2833-36a3-4cee-bc7e-93faf5c75c97 service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:46.069710 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f2833-36a3-4cee-bc7e-93faf5c75c97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:46.069992 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f2833-36a3-4cee-bc7e-93faf5c75c97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:46.073218 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-396f2833-36a3-4cee-bc7e-93faf5c75c97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:46.095542 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f2833-36a3-4cee-bc7e-93faf5c75c97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:46.097088 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f2833-36a3-4cee-bc7e-93faf5c75c97 service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:46.108088 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1309/2624] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:22:46.409720 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a701e-40eb-4f8d-985b-064ef9e17cc4 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:46.411271 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-786a701e-40eb-4f8d-985b-064ef9e17cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:46.418102 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:46.492963 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f2833-36a3-4cee-bc7e-93faf5c75c97 service neutron] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:46.494698 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-396f2833-36a3-4cee-bc7e-93faf5c75c97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:46.526201 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f2833-36a3-4cee-bc7e-93faf5c75c97 service neutron] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:46.535032 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16aeec0-cf95-4a97-bec2-c650c44ce953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:46.535253 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a701e-40eb-4f8d-985b-064ef9e17cc4 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:46.536439 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16aeec0-cf95-4a97-bec2-c650c44ce953 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:46.536883 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16aeec0-cf95-4a97-bec2-c650c44ce953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:46.537469 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16aeec0-cf95-4a97-bec2-c650c44ce953 admin admin] PATH_INFO: `/v3/users/ba0127dd69da407ab096773fc9e22530` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:46.539329 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16aeec0-cf95-4a97-bec2-c650c44ce953 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba0127dd69da407ab096773fc9e22530)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:46.541957 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16aeec0-cf95-4a97-bec2-c650c44ce953 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:46.542379 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32728d5-dea6-441a-856c-7b92f3b586e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0437c45d72cd4a38b9d48ee6f50946fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:46.543136 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32728d5-dea6-441a-856c-7b92f3b586e2 service placement] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:46.543631 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32728d5-dea6-441a-856c-7b92f3b586e2 service placement] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:46.543986 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32728d5-dea6-441a-856c-7b92f3b586e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:46.545976 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a32728d5-dea6-441a-856c-7b92f3b586e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:46.562875 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32728d5-dea6-441a-856c-7b92f3b586e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:46.565781 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32728d5-dea6-441a-856c-7b92f3b586e2 service placement] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:46.582669 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1316/2625] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:22:46.610527 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c16aeec0-cf95-4a97-bec2-c650c44ce953 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba0127dd69da407ab096773fc9e22530'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:46.621760 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c16aeec0-cf95-4a97-bec2-c650c44ce953 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba0127dd69da407ab096773fc9e22530'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:46.622388 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c16aeec0-cf95-4a97-bec2-c650c44ce953 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba0127dd69da407ab096773fc9e22530'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:46.623007 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c16aeec0-cf95-4a97-bec2-c650c44ce953 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba0127dd69da407ab096773fc9e22530'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:46.632440 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c16aeec0-cf95-4a97-bec2-c650c44ce953 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba0127dd69da407ab096773fc9e22530'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:46.660239 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1310/2626] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:22:46 2022] DELETE /identity/v3/users/ba0127dd69da407ab096773fc9e22530 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:22:46.872581 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32728d5-dea6-441a-856c-7b92f3b586e2 service placement] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:46.887380 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32728d5-dea6-441a-856c-7b92f3b586e2 service placement] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:46.952598 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a5bb1e-1c9a-436f-83c2-7985aae38d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:46.953922 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5bb1e-1c9a-436f-83c2-7985aae38d0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:46.954203 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5bb1e-1c9a-436f-83c2-7985aae38d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:46.954479 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5bb1e-1c9a-436f-83c2-7985aae38d0f admin admin] PATH_INFO: `/v3/projects/0cdaf724b92644a0ae7fbb71147aef40` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:46.955663 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a5bb1e-1c9a-436f-83c2-7985aae38d0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cdaf724b92644a0ae7fbb71147aef40)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:46.958093 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a5bb1e-1c9a-436f-83c2-7985aae38d0f admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:47.020859 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-88a5bb1e-1c9a-436f-83c2-7985aae38d0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cdaf724b92644a0ae7fbb71147aef40', 'request_id': 'req-88a5bb1e-1c9a-436f-83c2-7985aae38d0f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:47.030077 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-88a5bb1e-1c9a-436f-83c2-7985aae38d0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cdaf724b92644a0ae7fbb71147aef40', 'request_id': 'req-88a5bb1e-1c9a-436f-83c2-7985aae38d0f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:47.039993 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-88a5bb1e-1c9a-436f-83c2-7985aae38d0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cdaf724b92644a0ae7fbb71147aef40', 'request_id': 'req-88a5bb1e-1c9a-436f-83c2-7985aae38d0f'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:47.053956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-88a5bb1e-1c9a-436f-83c2-7985aae38d0f admin admin] The token cache is being invalidate because project 0cdaf724b92644a0ae7fbb71147aef40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:22:47.054581 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-88a5bb1e-1c9a-436f-83c2-7985aae38d0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:47.055315 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-88a5bb1e-1c9a-436f-83c2-7985aae38d0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:47.057029 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1317/2627] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:22:46 2022] DELETE /identity/v3/projects/0cdaf724b92644a0ae7fbb71147aef40 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:22:47.064391 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3729f-5786-4cae-8c0f-16686480d286 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:47.068369 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3729f-5786-4cae-8c0f-16686480d286 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:47.068810 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3729f-5786-4cae-8c0f-16686480d286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:47.088983 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-67c3729f-5786-4cae-8c0f-16686480d286 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:47.090454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-67c3729f-5786-4cae-8c0f-16686480d286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:47.161838 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1311/2628] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:47.170699 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8447b38-2a40-4a8c-a384-f44cc6f0a2b9 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:47.170998 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8447b38-2a40-4a8c-a384-f44cc6f0a2b9 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:47.171284 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8447b38-2a40-4a8c-a384-f44cc6f0a2b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:47.191922 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-c8447b38-2a40-4a8c-a384-f44cc6f0a2b9 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:47.193787 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-c8447b38-2a40-4a8c-a384-f44cc6f0a2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:47.204012 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1318/2629] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:22:47.209308 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c3729f-5786-4cae-8c0f-16686480d286 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:47.221521 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c3729f-5786-4cae-8c0f-16686480d286 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:47.227361 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f2798-bf77-48c7-8ced-a96689a29a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:47.228060 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f2798-bf77-48c7-8ced-a96689a29a9a admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:47.228346 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f2798-bf77-48c7-8ced-a96689a29a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:47.228623 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f2798-bf77-48c7-8ced-a96689a29a9a admin admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:47.229415 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7f2798-bf77-48c7-8ced-a96689a29a9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:22:47.229730 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7f2798-bf77-48c7-8ced-a96689a29a9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:47.231466 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7f2798-bf77-48c7-8ced-a96689a29a9a admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:47.252616 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1312/2630] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:22:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:22:47.254199 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8447b38-2a40-4a8c-a384-f44cc6f0a2b9 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:47.264711 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8447b38-2a40-4a8c-a384-f44cc6f0a2b9 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:47.269426 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdc6757-1c30-4f18-8bf6-f526439d59d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:47.270094 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc6757-1c30-4f18-8bf6-f526439d59d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:47.270376 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc6757-1c30-4f18-8bf6-f526439d59d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:47.270652 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc6757-1c30-4f18-8bf6-f526439d59d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:47.271353 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdc6757-1c30-4f18-8bf6-f526439d59d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:47.273489 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdc6757-1c30-4f18-8bf6-f526439d59d8 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:47.286951 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:47.300361 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1319/2631] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:22:47 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:22:47.304997 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f2798-bf77-48c7-8ced-a96689a29a9a admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:47.316297 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f2798-bf77-48c7-8ced-a96689a29a9a admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:47.369487 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:47.384357 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6431f-b161-40d6-b296-031fd553954a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:47.385115 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6431f-b161-40d6-b296-031fd553954a admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:47.385399 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6431f-b161-40d6-b296-031fd553954a admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:47.385667 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6431f-b161-40d6-b296-031fd553954a admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:47.386356 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce6431f-b161-40d6-b296-031fd553954a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:47.388272 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce6431f-b161-40d6-b296-031fd553954a admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:47.410415 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1313/2632] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:22:47 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:22:47.415604 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdc6757-1c30-4f18-8bf6-f526439d59d8 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:47.427772 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdc6757-1c30-4f18-8bf6-f526439d59d8 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:47.434816 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200625dd-fe82-48bb-a91a-d053d3941478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:47.435545 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200625dd-fe82-48bb-a91a-d053d3941478 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:47.435827 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200625dd-fe82-48bb-a91a-d053d3941478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:47.436096 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200625dd-fe82-48bb-a91a-d053d3941478 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:47.436868 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200625dd-fe82-48bb-a91a-d053d3941478 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:47.438527 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200625dd-fe82-48bb-a91a-d053d3941478 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:47.450374 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1320/2633] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:22:47.454880 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6431f-b161-40d6-b296-031fd553954a admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:47.465847 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6431f-b161-40d6-b296-031fd553954a admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:47.470437 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb1a5f-fc39-41c7-9b66-e32960455bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:47.471103 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb1a5f-fc39-41c7-9b66-e32960455bed admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:47.471386 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb1a5f-fc39-41c7-9b66-e32960455bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:47.471661 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb1a5f-fc39-41c7-9b66-e32960455bed admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:47.472395 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb1a5f-fc39-41c7-9b66-e32960455bed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:47.474042 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb1a5f-fc39-41c7-9b66-e32960455bed admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:47.486707 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1314/2634] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:22:47.492018 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200625dd-fe82-48bb-a91a-d053d3941478 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:47.504567 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200625dd-fe82-48bb-a91a-d053d3941478 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:47.511165 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e993d2-4814-4429-8cba-a51432129c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:47.512405 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e993d2-4814-4429-8cba-a51432129c66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:47.512685 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e993d2-4814-4429-8cba-a51432129c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:47.512998 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e993d2-4814-4429-8cba-a51432129c66 admin admin] PATH_INFO: `/v3/projects/083acc51fd7842fc893ebd00da3c64dc/users/0dd47d03d78a4349bb3fb3dc0a0100c5/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:47.529452 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e993d2-4814-4429-8cba-a51432129c66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=083acc51fd7842fc893ebd00da3c64dc, user_id=0dd47d03d78a4349bb3fb3dc0a0100c5, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:47.532291 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e993d2-4814-4429-8cba-a51432129c66 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:47.549048 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1321/2635] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:22:47 2022] PUT /identity/v3/projects/083acc51fd7842fc893ebd00da3c64dc/users/0dd47d03d78a4349bb3fb3dc0a0100c5/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:47.560387 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3788a3-9fcd-4229-923e-2b189017addb None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:47.560671 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3788a3-9fcd-4229-923e-2b189017addb None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:47.560988 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3788a3-9fcd-4229-923e-2b189017addb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:47.591537 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-ab3788a3-9fcd-4229-923e-2b189017addb None None] MFA Rules not processed for user `0dd47d03d78a4349bb3fb3dc0a0100c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:47.593348 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-ab3788a3-9fcd-4229-923e-2b189017addb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:47.685474 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1315/2636] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:47.710730 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e993d2-4814-4429-8cba-a51432129c66 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:47.712749 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-72e993d2-4814-4429-8cba-a51432129c66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:47.817151 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e993d2-4814-4429-8cba-a51432129c66 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:47.830081 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc76fff-2c06-431e-b458-9db9d6308484 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:47.831254 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc76fff-2c06-431e-b458-9db9d6308484 service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:47.831738 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc76fff-2c06-431e-b458-9db9d6308484 service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:47.832372 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc76fff-2c06-431e-b458-9db9d6308484 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:47.841746 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc76fff-2c06-431e-b458-9db9d6308484 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:47.843639 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc76fff-2c06-431e-b458-9db9d6308484 service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:47.855705 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1322/2637] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:47 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:22:50.631598 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3788a3-9fcd-4229-923e-2b189017addb None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:50.703157 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3788a3-9fcd-4229-923e-2b189017addb None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:50.708932 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6463313-20e0-44fd-955a-ca0452f010da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:50.710314 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6463313-20e0-44fd-955a-ca0452f010da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:50.710642 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6463313-20e0-44fd-955a-ca0452f010da admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:50.711168 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6463313-20e0-44fd-955a-ca0452f010da admin admin] PATH_INFO: `/v3/users/0dd47d03d78a4349bb3fb3dc0a0100c5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:50.712451 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6463313-20e0-44fd-955a-ca0452f010da admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dd47d03d78a4349bb3fb3dc0a0100c5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:50.714452 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6463313-20e0-44fd-955a-ca0452f010da admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:50.775556 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c6463313-20e0-44fd-955a-ca0452f010da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dd47d03d78a4349bb3fb3dc0a0100c5'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:50.788130 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c6463313-20e0-44fd-955a-ca0452f010da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dd47d03d78a4349bb3fb3dc0a0100c5'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:50.789847 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c6463313-20e0-44fd-955a-ca0452f010da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dd47d03d78a4349bb3fb3dc0a0100c5'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:50.790648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c6463313-20e0-44fd-955a-ca0452f010da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dd47d03d78a4349bb3fb3dc0a0100c5'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:50.793834 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:50.800455 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c6463313-20e0-44fd-955a-ca0452f010da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dd47d03d78a4349bb3fb3dc0a0100c5'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:50.825189 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1316/2638] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:22:50 2022] DELETE /identity/v3/users/0dd47d03d78a4349bb3fb3dc0a0100c5 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:50.830362 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ed033-f299-4a61-a4b7-71799592d8ac None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:50.830945 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ed033-f299-4a61-a4b7-71799592d8ac None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:50.831283 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ed033-f299-4a61-a4b7-71799592d8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:50.855471 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-b51ed033-f299-4a61-a4b7-71799592d8ac None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:50.857177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-b51ed033-f299-4a61-a4b7-71799592d8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:50.936644 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1323/2639] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:50.947821 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6463313-20e0-44fd-955a-ca0452f010da admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:50.949512 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-c6463313-20e0-44fd-955a-ca0452f010da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:51.021677 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6463313-20e0-44fd-955a-ca0452f010da admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:51.031178 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc8fd3-0c65-4f2e-b69a-6463326e3b16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:51.034535 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc8fd3-0c65-4f2e-b69a-6463326e3b16 service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:51.034535 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc8fd3-0c65-4f2e-b69a-6463326e3b16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:51.034535 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc8fd3-0c65-4f2e-b69a-6463326e3b16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:51.046283 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cc8fd3-0c65-4f2e-b69a-6463326e3b16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:51.047805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cc8fd3-0c65-4f2e-b69a-6463326e3b16 service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:51.058884 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1317/2640] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:22:51.413103 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ed033-f299-4a61-a4b7-71799592d8ac None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:51.423485 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ed033-f299-4a61-a4b7-71799592d8ac None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:51.431137 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35591d36-25dc-466a-80a3-344e1f82fd6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:51.431137 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35591d36-25dc-466a-80a3-344e1f82fd6d service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:51.431137 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35591d36-25dc-466a-80a3-344e1f82fd6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:51.431137 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35591d36-25dc-466a-80a3-344e1f82fd6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:51.433139 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-35591d36-25dc-466a-80a3-344e1f82fd6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:51.483998 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:51.504549 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35591d36-25dc-466a-80a3-344e1f82fd6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:51.506065 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35591d36-25dc-466a-80a3-344e1f82fd6d service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:51.517726 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1324/2641] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:22:51.865052 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc8fd3-0c65-4f2e-b69a-6463326e3b16 service neutron] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:51.866768 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-01cc8fd3-0c65-4f2e-b69a-6463326e3b16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:51.881090 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc8fd3-0c65-4f2e-b69a-6463326e3b16 service neutron] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:51.887280 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad68663-41d1-4232-99cf-f49e48668e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:51.888173 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad68663-41d1-4232-99cf-f49e48668e42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:51.888459 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad68663-41d1-4232-99cf-f49e48668e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:51.888736 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad68663-41d1-4232-99cf-f49e48668e42 admin admin] PATH_INFO: `/v3/projects/083acc51fd7842fc893ebd00da3c64dc` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:51.890288 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad68663-41d1-4232-99cf-f49e48668e42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=083acc51fd7842fc893ebd00da3c64dc)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:51.892090 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad68663-41d1-4232-99cf-f49e48668e42 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:51.944446 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-3ad68663-41d1-4232-99cf-f49e48668e42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '083acc51fd7842fc893ebd00da3c64dc', 'request_id': 'req-3ad68663-41d1-4232-99cf-f49e48668e42'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:51.952077 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-3ad68663-41d1-4232-99cf-f49e48668e42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '083acc51fd7842fc893ebd00da3c64dc', 'request_id': 'req-3ad68663-41d1-4232-99cf-f49e48668e42'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:51.963216 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-3ad68663-41d1-4232-99cf-f49e48668e42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '083acc51fd7842fc893ebd00da3c64dc', 'request_id': 'req-3ad68663-41d1-4232-99cf-f49e48668e42'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:51.974738 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-3ad68663-41d1-4232-99cf-f49e48668e42 admin admin] The token cache is being invalidate because project 083acc51fd7842fc893ebd00da3c64dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:22:51.975027 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-3ad68663-41d1-4232-99cf-f49e48668e42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:51.976117 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-3ad68663-41d1-4232-99cf-f49e48668e42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:51.977849 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1318/2642] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:22:51 2022] DELETE /identity/v3/projects/083acc51fd7842fc893ebd00da3c64dc => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:51.982981 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7484cb-fc16-4159-8fe5-88d1657a5827 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:51.983498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7484cb-fc16-4159-8fe5-88d1657a5827 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:51.983585 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7484cb-fc16-4159-8fe5-88d1657a5827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:52.006294 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-ba7484cb-fc16-4159-8fe5-88d1657a5827 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:52.007673 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ba7484cb-fc16-4159-8fe5-88d1657a5827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:52.091348 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1325/2643] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:52.093600 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d839be7-0549-4def-b97d-63877ba02564 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:52.093891 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d839be7-0549-4def-b97d-63877ba02564 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:52.094175 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d839be7-0549-4def-b97d-63877ba02564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:52.113235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-9d839be7-0549-4def-b97d-63877ba02564 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:52.114473 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-9d839be7-0549-4def-b97d-63877ba02564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:52.122816 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1319/2644] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:52.127336 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7484cb-fc16-4159-8fe5-88d1657a5827 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:52.138841 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7484cb-fc16-4159-8fe5-88d1657a5827 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:52.144578 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d03d1d-7909-4bf3-8d9e-a815ebf7b28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:52.145485 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d03d1d-7909-4bf3-8d9e-a815ebf7b28d admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:52.147059 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d03d1d-7909-4bf3-8d9e-a815ebf7b28d admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:52.147410 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d03d1d-7909-4bf3-8d9e-a815ebf7b28d admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:52.148274 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d03d1d-7909-4bf3-8d9e-a815ebf7b28d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:22:52.149107 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d03d1d-7909-4bf3-8d9e-a815ebf7b28d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:52.151643 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d03d1d-7909-4bf3-8d9e-a815ebf7b28d admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:52.174773 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1326/2645] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:22:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:22:52.180340 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d839be7-0549-4def-b97d-63877ba02564 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:52.192984 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d839be7-0549-4def-b97d-63877ba02564 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:52.199770 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da07a94-f728-448c-8cf8-b73bc1a32274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:52.200612 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da07a94-f728-448c-8cf8-b73bc1a32274 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:52.200863 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da07a94-f728-448c-8cf8-b73bc1a32274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:52.201477 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da07a94-f728-448c-8cf8-b73bc1a32274 admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:52.201933 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da07a94-f728-448c-8cf8-b73bc1a32274 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:52.204130 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da07a94-f728-448c-8cf8-b73bc1a32274 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:52.237900 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1320/2646] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:22:52 2022] POST /identity/v3/projects => generated 375 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:22:52.242554 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d03d1d-7909-4bf3-8d9e-a815ebf7b28d admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:52.253886 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d03d1d-7909-4bf3-8d9e-a815ebf7b28d admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:52.323786 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53220c-b167-4195-af39-ad641598fa07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:52.324611 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53220c-b167-4195-af39-ad641598fa07 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:52.324896 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53220c-b167-4195-af39-ad641598fa07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:52.325219 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53220c-b167-4195-af39-ad641598fa07 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:52.325912 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf53220c-b167-4195-af39-ad641598fa07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:52.327861 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf53220c-b167-4195-af39-ad641598fa07 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:52.349126 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1327/2647] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:22:52 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:22:52.353926 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da07a94-f728-448c-8cf8-b73bc1a32274 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:52.364692 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da07a94-f728-448c-8cf8-b73bc1a32274 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:52.369644 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94671505-c267-4008-b901-6fbbb3857be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:52.370323 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94671505-c267-4008-b901-6fbbb3857be9 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:52.370604 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94671505-c267-4008-b901-6fbbb3857be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:52.370877 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94671505-c267-4008-b901-6fbbb3857be9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:52.371661 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94671505-c267-4008-b901-6fbbb3857be9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:52.373301 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94671505-c267-4008-b901-6fbbb3857be9 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:52.385856 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1321/2648] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:22:52.390175 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53220c-b167-4195-af39-ad641598fa07 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:52.403002 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53220c-b167-4195-af39-ad641598fa07 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:52.409477 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fb973b-9923-438f-8fff-812baf38d30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:52.410200 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb973b-9923-438f-8fff-812baf38d30a admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:52.410466 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb973b-9923-438f-8fff-812baf38d30a admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:52.410802 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fb973b-9923-438f-8fff-812baf38d30a admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:52.411559 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fb973b-9923-438f-8fff-812baf38d30a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:52.413464 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fb973b-9923-438f-8fff-812baf38d30a admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:52.430003 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1328/2649] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:22:52.431539 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94671505-c267-4008-b901-6fbbb3857be9 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:52.452626 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94671505-c267-4008-b901-6fbbb3857be9 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:52.467287 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d4f34-1758-49d0-bc01-7945978bb6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:52.468207 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d4f34-1758-49d0-bc01-7945978bb6fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:52.468485 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d4f34-1758-49d0-bc01-7945978bb6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:52.468757 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d4f34-1758-49d0-bc01-7945978bb6fd admin admin] PATH_INFO: `/v3/projects/fce4740b494c472dba26d969ed39a7ee/users/6f07bcf30f81451a896d9f8f3820f3a4/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:52.491230 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109d4f34-1758-49d0-bc01-7945978bb6fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=fce4740b494c472dba26d969ed39a7ee, user_id=6f07bcf30f81451a896d9f8f3820f3a4, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:52.496423 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109d4f34-1758-49d0-bc01-7945978bb6fd admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:52.509313 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:52.519550 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1322/2650] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:22:52 2022] PUT /identity/v3/projects/fce4740b494c472dba26d969ed39a7ee/users/6f07bcf30f81451a896d9f8f3820f3a4/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:52.531664 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceafa01-9b16-41db-a230-469c0487e943 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:52.531955 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceafa01-9b16-41db-a230-469c0487e943 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:52.532259 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceafa01-9b16-41db-a230-469c0487e943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:52.563239 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-cceafa01-9b16-41db-a230-469c0487e943 None None] MFA Rules not processed for user `6f07bcf30f81451a896d9f8f3820f3a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:52.564452 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-cceafa01-9b16-41db-a230-469c0487e943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:52.625374 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:52.666813 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1329/2651] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:52.681742 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d4f34-1758-49d0-bc01-7945978bb6fd admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:52.683834 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-109d4f34-1758-49d0-bc01-7945978bb6fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:52.787563 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d4f34-1758-49d0-bc01-7945978bb6fd admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:52.794338 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ad8d0-ee58-44a6-96d6-0f668d56947c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:52.795025 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ad8d0-ee58-44a6-96d6-0f668d56947c service nova] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:52.795310 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ad8d0-ee58-44a6-96d6-0f668d56947c service nova] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:52.795590 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ad8d0-ee58-44a6-96d6-0f668d56947c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:52.806164 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ad8d0-ee58-44a6-96d6-0f668d56947c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:52.807864 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ad8d0-ee58-44a6-96d6-0f668d56947c service nova] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:52.818222 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1323/2652] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:22:53.051574 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceafa01-9b16-41db-a230-469c0487e943 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:53.124877 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceafa01-9b16-41db-a230-469c0487e943 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:53.129939 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841fcc2b-fa47-4bfd-9c70-3288bc45b053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:53.130835 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841fcc2b-fa47-4bfd-9c70-3288bc45b053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:53.131109 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841fcc2b-fa47-4bfd-9c70-3288bc45b053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:53.131411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841fcc2b-fa47-4bfd-9c70-3288bc45b053 admin admin] PATH_INFO: `/v3/users/6f07bcf30f81451a896d9f8f3820f3a4` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:53.132857 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841fcc2b-fa47-4bfd-9c70-3288bc45b053 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f07bcf30f81451a896d9f8f3820f3a4)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:53.134894 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841fcc2b-fa47-4bfd-9c70-3288bc45b053 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:53.185758 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-841fcc2b-fa47-4bfd-9c70-3288bc45b053 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f07bcf30f81451a896d9f8f3820f3a4'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:53.196499 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-841fcc2b-fa47-4bfd-9c70-3288bc45b053 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f07bcf30f81451a896d9f8f3820f3a4'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:53.197261 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-841fcc2b-fa47-4bfd-9c70-3288bc45b053 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f07bcf30f81451a896d9f8f3820f3a4'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:53.197883 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-841fcc2b-fa47-4bfd-9c70-3288bc45b053 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f07bcf30f81451a896d9f8f3820f3a4'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:53.213859 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-841fcc2b-fa47-4bfd-9c70-3288bc45b053 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f07bcf30f81451a896d9f8f3820f3a4'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:53.234289 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1330/2653] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:22:53 2022] DELETE /identity/v3/users/6f07bcf30f81451a896d9f8f3820f3a4 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:53.239139 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dca689-0f3c-4a89-8146-148656db82b7 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:53.239700 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dca689-0f3c-4a89-8146-148656db82b7 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:53.239700 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dca689-0f3c-4a89-8146-148656db82b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:53.258304 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-21dca689-0f3c-4a89-8146-148656db82b7 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:53.259536 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-21dca689-0f3c-4a89-8146-148656db82b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:53.330209 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1324/2654] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:53.342381 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841fcc2b-fa47-4bfd-9c70-3288bc45b053 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:53.343866 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-841fcc2b-fa47-4bfd-9c70-3288bc45b053 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:53.408914 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841fcc2b-fa47-4bfd-9c70-3288bc45b053 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:53.413477 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7926ed34-ebb5-45b1-989c-df35c5fc58c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:53.414474 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926ed34-ebb5-45b1-989c-df35c5fc58c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:53.414474 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926ed34-ebb5-45b1-989c-df35c5fc58c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:53.414695 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926ed34-ebb5-45b1-989c-df35c5fc58c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:53.424762 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7926ed34-ebb5-45b1-989c-df35c5fc58c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:53.426254 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7926ed34-ebb5-45b1-989c-df35c5fc58c3 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:53.432310 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:53.437986 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1331/2655] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:22:53.814745 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dca689-0f3c-4a89-8146-148656db82b7 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:53.816353 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-21dca689-0f3c-4a89-8146-148656db82b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:53.829202 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dca689-0f3c-4a89-8146-148656db82b7 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:53.843752 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7691c9ae-b491-4036-ab6a-1f6b66a1bd98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:53.844671 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691c9ae-b491-4036-ab6a-1f6b66a1bd98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:53.844671 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691c9ae-b491-4036-ab6a-1f6b66a1bd98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:53.844671 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691c9ae-b491-4036-ab6a-1f6b66a1bd98 admin admin] PATH_INFO: `/v3/projects/fce4740b494c472dba26d969ed39a7ee` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:53.847628 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7691c9ae-b491-4036-ab6a-1f6b66a1bd98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fce4740b494c472dba26d969ed39a7ee)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:53.848081 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7691c9ae-b491-4036-ab6a-1f6b66a1bd98 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:53.910093 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-7691c9ae-b491-4036-ab6a-1f6b66a1bd98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fce4740b494c472dba26d969ed39a7ee', 'request_id': 'req-7691c9ae-b491-4036-ab6a-1f6b66a1bd98'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:53.917505 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-7691c9ae-b491-4036-ab6a-1f6b66a1bd98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fce4740b494c472dba26d969ed39a7ee', 'request_id': 'req-7691c9ae-b491-4036-ab6a-1f6b66a1bd98'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:53.926830 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-7691c9ae-b491-4036-ab6a-1f6b66a1bd98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fce4740b494c472dba26d969ed39a7ee', 'request_id': 'req-7691c9ae-b491-4036-ab6a-1f6b66a1bd98'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:53.944604 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-7691c9ae-b491-4036-ab6a-1f6b66a1bd98 admin admin] The token cache is being invalidate because project fce4740b494c472dba26d969ed39a7ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:22:53.945157 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-7691c9ae-b491-4036-ab6a-1f6b66a1bd98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:53.946041 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-7691c9ae-b491-4036-ab6a-1f6b66a1bd98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:22:53.947656 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1325/2656] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:22:53 2022] DELETE /identity/v3/projects/fce4740b494c472dba26d969ed39a7ee => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:53.953626 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc56183-603c-4c57-a7d6-825476ed13f5 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:53.954179 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc56183-603c-4c57-a7d6-825476ed13f5 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:53.954459 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc56183-603c-4c57-a7d6-825476ed13f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:53.984787 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-0dc56183-603c-4c57-a7d6-825476ed13f5 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:53.986981 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-0dc56183-603c-4c57-a7d6-825476ed13f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:54.144325 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1332/2657] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:54.146247 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1781f5-9c44-436f-aba7-ff7f9bb77f54 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:54.146689 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1781f5-9c44-436f-aba7-ff7f9bb77f54 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:54.147163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1781f5-9c44-436f-aba7-ff7f9bb77f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:54.164481 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:54.175106 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-8c1781f5-9c44-436f-aba7-ff7f9bb77f54 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:54.177444 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-8c1781f5-9c44-436f-aba7-ff7f9bb77f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:54.186732 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1326/2658] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:22:54.191362 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc56183-603c-4c57-a7d6-825476ed13f5 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:54.203930 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc56183-603c-4c57-a7d6-825476ed13f5 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:54.211054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d7776-8a64-46e2-adad-0a6ea0046285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:54.211778 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d7776-8a64-46e2-adad-0a6ea0046285 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:54.212058 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d7776-8a64-46e2-adad-0a6ea0046285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:54.212330 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d7776-8a64-46e2-adad-0a6ea0046285 admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:54.213810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75d7776-8a64-46e2-adad-0a6ea0046285 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:22:54.214358 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75d7776-8a64-46e2-adad-0a6ea0046285 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:54.217022 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75d7776-8a64-46e2-adad-0a6ea0046285 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:54.234204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1333/2659] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:22:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:22:54.239171 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1781f5-9c44-436f-aba7-ff7f9bb77f54 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:54.260068 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1781f5-9c44-436f-aba7-ff7f9bb77f54 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:54.272193 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05f70ab-8097-478c-9b14-c17c8122f2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:54.274592 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f70ab-8097-478c-9b14-c17c8122f2f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:54.275043 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f70ab-8097-478c-9b14-c17c8122f2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:54.275983 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05f70ab-8097-478c-9b14-c17c8122f2f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:54.277239 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05f70ab-8097-478c-9b14-c17c8122f2f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:54.282895 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05f70ab-8097-478c-9b14-c17c8122f2f6 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:54.342086 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1327/2660] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:22:54 2022] POST /identity/v3/projects => generated 373 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:22:54.342870 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d7776-8a64-46e2-adad-0a6ea0046285 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:54.361397 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d7776-8a64-46e2-adad-0a6ea0046285 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:54.429246 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd228d1c-4bef-4202-a538-013e180d6da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:54.429940 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd228d1c-4bef-4202-a538-013e180d6da1 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:54.430222 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd228d1c-4bef-4202-a538-013e180d6da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:54.430527 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd228d1c-4bef-4202-a538-013e180d6da1 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:54.431236 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd228d1c-4bef-4202-a538-013e180d6da1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:54.433257 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd228d1c-4bef-4202-a538-013e180d6da1 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:54.457845 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1334/2661] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:22:54 2022] POST /identity/v3/users => generated 287 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:22:54.462643 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05f70ab-8097-478c-9b14-c17c8122f2f6 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:54.473903 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05f70ab-8097-478c-9b14-c17c8122f2f6 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:54.478990 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083dd75d-eea9-4424-8072-4dcce31a8842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:54.479685 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083dd75d-eea9-4424-8072-4dcce31a8842 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:54.479965 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083dd75d-eea9-4424-8072-4dcce31a8842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:54.480237 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083dd75d-eea9-4424-8072-4dcce31a8842 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:54.481019 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083dd75d-eea9-4424-8072-4dcce31a8842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:54.482654 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083dd75d-eea9-4424-8072-4dcce31a8842 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:54.494432 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1328/2662] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:22:54.500207 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd228d1c-4bef-4202-a538-013e180d6da1 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:54.512259 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd228d1c-4bef-4202-a538-013e180d6da1 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:54.518111 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c66af4-cce5-4533-a857-777d584b3035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:54.518812 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c66af4-cce5-4533-a857-777d584b3035 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:54.519101 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c66af4-cce5-4533-a857-777d584b3035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:54.519380 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c66af4-cce5-4533-a857-777d584b3035 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:54.520217 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c66af4-cce5-4533-a857-777d584b3035 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:54.521914 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c66af4-cce5-4533-a857-777d584b3035 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:54.536424 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1335/2663] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:22:54.541288 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083dd75d-eea9-4424-8072-4dcce31a8842 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:54.553107 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083dd75d-eea9-4424-8072-4dcce31a8842 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:54.559791 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b937a1ba-bd0a-4f19-b3fa-04d374f34f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:54.561138 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937a1ba-bd0a-4f19-b3fa-04d374f34f1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:54.561485 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937a1ba-bd0a-4f19-b3fa-04d374f34f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:54.561823 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937a1ba-bd0a-4f19-b3fa-04d374f34f1f admin admin] PATH_INFO: `/v3/projects/c6a34e9da689495093767d8221a778a2/users/14dea9f30b9e4018a759ec452e19a3a6/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:54.582348 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b937a1ba-bd0a-4f19-b3fa-04d374f34f1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6a34e9da689495093767d8221a778a2, user_id=14dea9f30b9e4018a759ec452e19a3a6, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:54.585385 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b937a1ba-bd0a-4f19-b3fa-04d374f34f1f admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:54.604991 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1329/2664] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:22:54 2022] PUT /identity/v3/projects/c6a34e9da689495093767d8221a778a2/users/14dea9f30b9e4018a759ec452e19a3a6/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:54.622028 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59072f-e7e9-419e-927e-977b8924c1c6 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:54.622401 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59072f-e7e9-419e-927e-977b8924c1c6 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:54.622690 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59072f-e7e9-419e-927e-977b8924c1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:54.666084 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-fd59072f-e7e9-419e-927e-977b8924c1c6 None None] MFA Rules not processed for user `14dea9f30b9e4018a759ec452e19a3a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:54.668214 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-fd59072f-e7e9-419e-927e-977b8924c1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:54.759791 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:54.777342 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1336/2665] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:22:54.778559 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b937a1ba-bd0a-4f19-b3fa-04d374f34f1f admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:54.876999 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b937a1ba-bd0a-4f19-b3fa-04d374f34f1f admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:54.882167 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a8724-0e7a-4499-8058-5f0937b0fac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:54.883095 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a8724-0e7a-4499-8058-5f0937b0fac4 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:54.883504 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a8724-0e7a-4499-8058-5f0937b0fac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:54.883856 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a8724-0e7a-4499-8058-5f0937b0fac4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:54.884902 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a8724-0e7a-4499-8058-5f0937b0fac4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:54.890321 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a8724-0e7a-4499-8058-5f0937b0fac4 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:54.928697 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1330/2666] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:22:54 2022] POST /identity/v3/projects => generated 373 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:22:54.935082 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd59072f-e7e9-419e-927e-977b8924c1c6 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:54.950793 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd59072f-e7e9-419e-927e-977b8924c1c6 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:55.026391 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df59554-1f93-468c-9232-59297d1801c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:55.027085 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df59554-1f93-468c-9232-59297d1801c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:55.027369 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df59554-1f93-468c-9232-59297d1801c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:55.027643 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df59554-1f93-468c-9232-59297d1801c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:55.029240 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df59554-1f93-468c-9232-59297d1801c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:55.031212 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df59554-1f93-468c-9232-59297d1801c4 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:55.060983 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1337/2667] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:22:54 2022] POST /identity/v3/users => generated 293 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:22:55.066405 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a8724-0e7a-4499-8058-5f0937b0fac4 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:55.082598 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a8724-0e7a-4499-8058-5f0937b0fac4 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:55.089081 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c802fa-0d92-4e6e-9666-12896335672b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:55.089799 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c802fa-0d92-4e6e-9666-12896335672b admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:55.090222 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c802fa-0d92-4e6e-9666-12896335672b admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:55.090579 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c802fa-0d92-4e6e-9666-12896335672b admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:55.091248 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c802fa-0d92-4e6e-9666-12896335672b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:55.093013 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c802fa-0d92-4e6e-9666-12896335672b admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:55.116663 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1331/2668] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:55 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:22:55.120962 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df59554-1f93-468c-9232-59297d1801c4 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:55.141081 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df59554-1f93-468c-9232-59297d1801c4 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:55.143633 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1124847-1ccb-4797-9db4-bf6c3f153aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:55.144953 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1124847-1ccb-4797-9db4-bf6c3f153aef admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:55.145553 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1124847-1ccb-4797-9db4-bf6c3f153aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:55.145652 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1124847-1ccb-4797-9db4-bf6c3f153aef admin admin] PATH_INFO: `/v3/projects/14daa6943ecb4bff8656ef740fe6f524/users/33e6364502124e468e2ce259cce96a6e/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:55.170153 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1124847-1ccb-4797-9db4-bf6c3f153aef admin admin] RBAC: Authorizing `identity:create_grant(project_id=14daa6943ecb4bff8656ef740fe6f524, user_id=33e6364502124e468e2ce259cce96a6e, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:55.173054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1124847-1ccb-4797-9db4-bf6c3f153aef admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:55.195942 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1338/2669] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:22:55 2022] PUT /identity/v3/projects/14daa6943ecb4bff8656ef740fe6f524/users/33e6364502124e468e2ce259cce96a6e/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:22:55.198759 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c802fa-0d92-4e6e-9666-12896335672b admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:55.217752 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c802fa-0d92-4e6e-9666-12896335672b admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:55.293930 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c2eb99-ad41-4435-81de-0d58cd1255a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:55.295329 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2eb99-ad41-4435-81de-0d58cd1255a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:55.295606 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2eb99-ad41-4435-81de-0d58cd1255a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:55.296040 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2eb99-ad41-4435-81de-0d58cd1255a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:55.296982 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c2eb99-ad41-4435-81de-0d58cd1255a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:55.298844 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c2eb99-ad41-4435-81de-0d58cd1255a1 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:55.318867 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1332/2670] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:22:55 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:22:55.324770 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1124847-1ccb-4797-9db4-bf6c3f153aef admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:55.336299 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1124847-1ccb-4797-9db4-bf6c3f153aef admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:55.341387 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcbc564-65fe-4ee8-af8a-5ae481e77241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:55.342303 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcbc564-65fe-4ee8-af8a-5ae481e77241 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:55.342583 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcbc564-65fe-4ee8-af8a-5ae481e77241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:55.342852 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcbc564-65fe-4ee8-af8a-5ae481e77241 admin admin] PATH_INFO: `/v3/projects/14daa6943ecb4bff8656ef740fe6f524/users/33e6364502124e468e2ce259cce96a6e/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:55.344978 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcbc564-65fe-4ee8-af8a-5ae481e77241 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14daa6943ecb4bff8656ef740fe6f524, user_id=33e6364502124e468e2ce259cce96a6e, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:55.347692 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcbc564-65fe-4ee8-af8a-5ae481e77241 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:55.360344 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1339/2671] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:22:55 2022] PUT /identity/v3/projects/14daa6943ecb4bff8656ef740fe6f524/users/33e6364502124e468e2ce259cce96a6e/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:22:55.373304 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c73c26-f26e-4b62-9df0-b6bd1438a4f0 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:55.373685 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c73c26-f26e-4b62-9df0-b6bd1438a4f0 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:55.373994 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c73c26-f26e-4b62-9df0-b6bd1438a4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:55.417548 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:22:55.423739 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-34c73c26-f26e-4b62-9df0-b6bd1438a4f0 None None] MFA Rules not processed for user `33e6364502124e468e2ce259cce96a6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:22:55.427781 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-34c73c26-f26e-4b62-9df0-b6bd1438a4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:55.524853 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1333/2672] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:22:55 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:22:55.536636 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcbc564-65fe-4ee8-af8a-5ae481e77241 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:55.539168 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-bdcbc564-65fe-4ee8-af8a-5ae481e77241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:22:55.622322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcbc564-65fe-4ee8-af8a-5ae481e77241 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:55.629005 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe7064-f91a-47ea-a8b6-12eb29be6e9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:55.629005 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe7064-f91a-47ea-a8b6-12eb29be6e9f service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:55.629368 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe7064-f91a-47ea-a8b6-12eb29be6e9f service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:55.629581 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe7064-f91a-47ea-a8b6-12eb29be6e9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:55.643407 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fe7064-f91a-47ea-a8b6-12eb29be6e9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:55.645283 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fe7064-f91a-47ea-a8b6-12eb29be6e9f service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:55.659405 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1340/2673] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:55 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:22:55.787410 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c73c26-f26e-4b62-9df0-b6bd1438a4f0 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:22:55.798789 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c73c26-f26e-4b62-9df0-b6bd1438a4f0 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:22:55.803366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80be1e1-8401-415c-bc27-36ce08e2fb4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:22:55.804293 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80be1e1-8401-415c-bc27-36ce08e2fb4b service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:22:55.804293 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80be1e1-8401-415c-bc27-36ce08e2fb4b service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:22:55.805240 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80be1e1-8401-415c-bc27-36ce08e2fb4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:22:55.921508 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80be1e1-8401-415c-bc27-36ce08e2fb4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:22:55.923457 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80be1e1-8401-415c-bc27-36ce08e2fb4b service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:22:55.936800 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1334/2674] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:22:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:23:01.161298 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe7064-f91a-47ea-a8b6-12eb29be6e9f service cinder] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:01.163879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-22fe7064-f91a-47ea-a8b6-12eb29be6e9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:01.237906 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe7064-f91a-47ea-a8b6-12eb29be6e9f service cinder] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:01.241851 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16323784-e9c7-44b1-a49f-32254c3081c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:01.242491 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16323784-e9c7-44b1-a49f-32254c3081c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:01.242836 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16323784-e9c7-44b1-a49f-32254c3081c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:01.243136 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16323784-e9c7-44b1-a49f-32254c3081c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:01.245098 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-16323784-e9c7-44b1-a49f-32254c3081c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:01.250182 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:01.313430 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16323784-e9c7-44b1-a49f-32254c3081c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:01.314787 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16323784-e9c7-44b1-a49f-32254c3081c1 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:01.324904 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1341/2675] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:23:05.930837 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=56 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:08.284299 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80be1e1-8401-415c-bc27-36ce08e2fb4b service cinder] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:08.285878 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-d80be1e1-8401-415c-bc27-36ce08e2fb4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:08.304525 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80be1e1-8401-415c-bc27-36ce08e2fb4b service cinder] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:08.309199 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50db116-21d2-468f-8249-eae1d2b9e55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:08.310092 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50db116-21d2-468f-8249-eae1d2b9e55c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:08.310367 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50db116-21d2-468f-8249-eae1d2b9e55c admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:08.310638 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50db116-21d2-468f-8249-eae1d2b9e55c admin admin] PATH_INFO: `/v3/users/4e8428365f1148a2934dd4596deeb43a` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:08.311821 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50db116-21d2-468f-8249-eae1d2b9e55c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e8428365f1148a2934dd4596deeb43a)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:08.313836 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50db116-21d2-468f-8249-eae1d2b9e55c admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:08.364199 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-f50db116-21d2-468f-8249-eae1d2b9e55c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e8428365f1148a2934dd4596deeb43a'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:08.371901 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-f50db116-21d2-468f-8249-eae1d2b9e55c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e8428365f1148a2934dd4596deeb43a'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:08.372618 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-f50db116-21d2-468f-8249-eae1d2b9e55c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e8428365f1148a2934dd4596deeb43a'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:08.373310 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-f50db116-21d2-468f-8249-eae1d2b9e55c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e8428365f1148a2934dd4596deeb43a'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:08.382648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-f50db116-21d2-468f-8249-eae1d2b9e55c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e8428365f1148a2934dd4596deeb43a'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:08.415494 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1335/2676] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:23:08 2022] DELETE /identity/v3/users/4e8428365f1148a2934dd4596deeb43a => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:08.608217 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16323784-e9c7-44b1-a49f-32254c3081c1 service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:08.618982 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16323784-e9c7-44b1-a49f-32254c3081c1 service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:08.683167 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b27840-6a97-4cc9-9020-aafeb26978ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:08.684037 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b27840-6a97-4cc9-9020-aafeb26978ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:08.684315 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b27840-6a97-4cc9-9020-aafeb26978ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:08.684588 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b27840-6a97-4cc9-9020-aafeb26978ef admin admin] PATH_INFO: `/v3/projects/f484976e63ce44daa10eadd2a9d6cc76` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:08.686125 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b27840-6a97-4cc9-9020-aafeb26978ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=f484976e63ce44daa10eadd2a9d6cc76)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:08.688081 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b27840-6a97-4cc9-9020-aafeb26978ef admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:08.741031 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-42b27840-6a97-4cc9-9020-aafeb26978ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f484976e63ce44daa10eadd2a9d6cc76', 'request_id': 'req-42b27840-6a97-4cc9-9020-aafeb26978ef'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:08.748955 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-42b27840-6a97-4cc9-9020-aafeb26978ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f484976e63ce44daa10eadd2a9d6cc76', 'request_id': 'req-42b27840-6a97-4cc9-9020-aafeb26978ef'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:08.761394 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-42b27840-6a97-4cc9-9020-aafeb26978ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f484976e63ce44daa10eadd2a9d6cc76', 'request_id': 'req-42b27840-6a97-4cc9-9020-aafeb26978ef'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:08.779073 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-42b27840-6a97-4cc9-9020-aafeb26978ef admin admin] The token cache is being invalidate because project f484976e63ce44daa10eadd2a9d6cc76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:23:08.779364 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-42b27840-6a97-4cc9-9020-aafeb26978ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:08.780028 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-42b27840-6a97-4cc9-9020-aafeb26978ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:08.781642 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1342/2677] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:23:08 2022] DELETE /identity/v3/projects/f484976e63ce44daa10eadd2a9d6cc76 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:08.785697 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50db116-21d2-468f-8249-eae1d2b9e55c admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:08.787157 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-f50db116-21d2-468f-8249-eae1d2b9e55c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:08.801163 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50db116-21d2-468f-8249-eae1d2b9e55c admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:08.863832 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced76af-bd78-42ef-9d50-785236bba680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:08.864905 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced76af-bd78-42ef-9d50-785236bba680 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:08.865227 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced76af-bd78-42ef-9d50-785236bba680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:08.865496 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced76af-bd78-42ef-9d50-785236bba680 admin admin] PATH_INFO: `/v3/users/fce6f6160c2e4021ba36f0deb035446e` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:08.866873 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced76af-bd78-42ef-9d50-785236bba680 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fce6f6160c2e4021ba36f0deb035446e)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:08.868811 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced76af-bd78-42ef-9d50-785236bba680 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:08.925347 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-8ced76af-bd78-42ef-9d50-785236bba680 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fce6f6160c2e4021ba36f0deb035446e'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:08.938537 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-8ced76af-bd78-42ef-9d50-785236bba680 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fce6f6160c2e4021ba36f0deb035446e'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:08.939693 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-8ced76af-bd78-42ef-9d50-785236bba680 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fce6f6160c2e4021ba36f0deb035446e'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:08.940681 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-8ced76af-bd78-42ef-9d50-785236bba680 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fce6f6160c2e4021ba36f0deb035446e'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:08.952258 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-8ced76af-bd78-42ef-9d50-785236bba680 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fce6f6160c2e4021ba36f0deb035446e'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:08.964717 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:08.979413 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1336/2678] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:23:08 2022] DELETE /identity/v3/users/fce6f6160c2e4021ba36f0deb035446e => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:09.169155 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b27840-6a97-4cc9-9020-aafeb26978ef admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:09.170641 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-42b27840-6a97-4cc9-9020-aafeb26978ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:09.190318 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b27840-6a97-4cc9-9020-aafeb26978ef admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:09.227446 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:09.263059 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb114e1-f9e0-4ef1-9a17-f4af77650454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:09.263942 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb114e1-f9e0-4ef1-9a17-f4af77650454 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:09.264230 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb114e1-f9e0-4ef1-9a17-f4af77650454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:09.264568 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb114e1-f9e0-4ef1-9a17-f4af77650454 admin admin] PATH_INFO: `/v3/projects/d2507516008649269ca7932a008d7db3` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:09.265966 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb114e1-f9e0-4ef1-9a17-f4af77650454 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2507516008649269ca7932a008d7db3)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:09.268004 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb114e1-f9e0-4ef1-9a17-f4af77650454 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:09.320669 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-3cb114e1-f9e0-4ef1-9a17-f4af77650454 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2507516008649269ca7932a008d7db3', 'request_id': 'req-3cb114e1-f9e0-4ef1-9a17-f4af77650454'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:09.327533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-3cb114e1-f9e0-4ef1-9a17-f4af77650454 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2507516008649269ca7932a008d7db3', 'request_id': 'req-3cb114e1-f9e0-4ef1-9a17-f4af77650454'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:09.342482 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-3cb114e1-f9e0-4ef1-9a17-f4af77650454 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2507516008649269ca7932a008d7db3', 'request_id': 'req-3cb114e1-f9e0-4ef1-9a17-f4af77650454'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:09.366480 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715dd3a-7c43-464d-8ade-628722683b1f None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:09.366964 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715dd3a-7c43-464d-8ade-628722683b1f None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:09.367062 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715dd3a-7c43-464d-8ade-628722683b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:09.367424 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-3cb114e1-f9e0-4ef1-9a17-f4af77650454 admin admin] The token cache is being invalidate because project d2507516008649269ca7932a008d7db3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:23:09.367782 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-3cb114e1-f9e0-4ef1-9a17-f4af77650454 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:09.369445 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-3cb114e1-f9e0-4ef1-9a17-f4af77650454 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:09.371422 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1343/2679] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:23:09 2022] DELETE /identity/v3/projects/d2507516008649269ca7932a008d7db3 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:09.390108 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-e715dd3a-7c43-464d-8ade-628722683b1f None None] MFA Rules not processed for user `e0e171895ccc4f1cbc69ead21175fc5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:09.391395 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-e715dd3a-7c43-464d-8ade-628722683b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:09.476441 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1337/2680] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:23:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:14.179032 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb114e1-f9e0-4ef1-9a17-f4af77650454 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:14.180626 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-3cb114e1-f9e0-4ef1-9a17-f4af77650454 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:14.254716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb114e1-f9e0-4ef1-9a17-f4af77650454 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:14.260141 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c9b40-017b-4e28-a84e-f65c0614b736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:14.261193 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c9b40-017b-4e28-a84e-f65c0614b736 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:14.261327 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c9b40-017b-4e28-a84e-f65c0614b736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:14.261607 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c9b40-017b-4e28-a84e-f65c0614b736 admin admin] PATH_INFO: `/v3/users/0c9f24f0f2ba401ba4cfd4bb6787c888` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:14.262964 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0c9b40-017b-4e28-a84e-f65c0614b736 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c9f24f0f2ba401ba4cfd4bb6787c888)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:14.265166 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0c9b40-017b-4e28-a84e-f65c0614b736 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:14.322186 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ac0c9b40-017b-4e28-a84e-f65c0614b736 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c9f24f0f2ba401ba4cfd4bb6787c888'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:14.329369 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ac0c9b40-017b-4e28-a84e-f65c0614b736 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c9f24f0f2ba401ba4cfd4bb6787c888'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:14.330131 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ac0c9b40-017b-4e28-a84e-f65c0614b736 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c9f24f0f2ba401ba4cfd4bb6787c888'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:14.330894 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ac0c9b40-017b-4e28-a84e-f65c0614b736 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c9f24f0f2ba401ba4cfd4bb6787c888'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:14.339335 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-ac0c9b40-017b-4e28-a84e-f65c0614b736 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c9f24f0f2ba401ba4cfd4bb6787c888'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:14.340592 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:14.359646 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1344/2681] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:23:14 2022] DELETE /identity/v3/users/0c9f24f0f2ba401ba4cfd4bb6787c888 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:14.364330 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9b087-561b-4109-bed1-98f61bf9118d None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:14.364839 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9b087-561b-4109-bed1-98f61bf9118d None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:14.364839 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9b087-561b-4109-bed1-98f61bf9118d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:14.383807 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-4df9b087-561b-4109-bed1-98f61bf9118d None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:14.385075 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-4df9b087-561b-4109-bed1-98f61bf9118d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:14.459068 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1338/2682] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:23:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:14.470239 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c9b40-017b-4e28-a84e-f65c0614b736 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:14.471803 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ac0c9b40-017b-4e28-a84e-f65c0614b736 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:14.533024 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c9b40-017b-4e28-a84e-f65c0614b736 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:14.536822 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e91c04-639e-4d18-a92b-a32019d11155 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:14.537513 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e91c04-639e-4d18-a92b-a32019d11155 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:14.538603 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e91c04-639e-4d18-a92b-a32019d11155 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:14.538897 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e91c04-639e-4d18-a92b-a32019d11155 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:14.549222 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e91c04-639e-4d18-a92b-a32019d11155 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:14.550696 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e91c04-639e-4d18-a92b-a32019d11155 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:14.561970 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1345/2683] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:23:15.252622 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df9b087-561b-4109-bed1-98f61bf9118d None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:15.254480 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-4df9b087-561b-4109-bed1-98f61bf9118d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:15.270032 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df9b087-561b-4109-bed1-98f61bf9118d None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:15.276463 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167ce5d-d33e-439f-9724-b32f06a3de6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:15.277576 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167ce5d-d33e-439f-9724-b32f06a3de6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:15.277867 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167ce5d-d33e-439f-9724-b32f06a3de6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:15.278139 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167ce5d-d33e-439f-9724-b32f06a3de6c admin admin] PATH_INFO: `/v3/projects/1a1c6bca01a94d56a5e8733239eda8aa` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:15.279494 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7167ce5d-d33e-439f-9724-b32f06a3de6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a1c6bca01a94d56a5e8733239eda8aa)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:15.282050 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7167ce5d-d33e-439f-9724-b32f06a3de6c admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:15.334418 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-7167ce5d-d33e-439f-9724-b32f06a3de6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a1c6bca01a94d56a5e8733239eda8aa', 'request_id': 'req-7167ce5d-d33e-439f-9724-b32f06a3de6c'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:15.340837 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-7167ce5d-d33e-439f-9724-b32f06a3de6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a1c6bca01a94d56a5e8733239eda8aa', 'request_id': 'req-7167ce5d-d33e-439f-9724-b32f06a3de6c'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:15.348818 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-7167ce5d-d33e-439f-9724-b32f06a3de6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a1c6bca01a94d56a5e8733239eda8aa', 'request_id': 'req-7167ce5d-d33e-439f-9724-b32f06a3de6c'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:15.359789 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-7167ce5d-d33e-439f-9724-b32f06a3de6c admin admin] The token cache is being invalidate because project 1a1c6bca01a94d56a5e8733239eda8aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:23:15.360082 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-7167ce5d-d33e-439f-9724-b32f06a3de6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:15.360730 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-7167ce5d-d33e-439f-9724-b32f06a3de6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:15.362354 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1339/2684] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:23:15 2022] DELETE /identity/v3/projects/1a1c6bca01a94d56a5e8733239eda8aa => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:15.368095 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc46b8-2484-450b-b882-bedc472e857f None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:15.368648 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc46b8-2484-450b-b882-bedc472e857f None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:15.368648 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc46b8-2484-450b-b882-bedc472e857f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:15.387561 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-4dcc46b8-2484-450b-b882-bedc472e857f None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:15.388789 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-4dcc46b8-2484-450b-b882-bedc472e857f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:15.454673 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1346/2685] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:23:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:15.462839 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5079b18-e947-4abe-bd03-f48e5ff6b75a None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:15.463358 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5079b18-e947-4abe-bd03-f48e5ff6b75a None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:15.463492 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5079b18-e947-4abe-bd03-f48e5ff6b75a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:15.483413 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-c5079b18-e947-4abe-bd03-f48e5ff6b75a None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:15.486482 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-c5079b18-e947-4abe-bd03-f48e5ff6b75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:15.494788 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1340/2686] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:23:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:15.499519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcc46b8-2484-450b-b882-bedc472e857f None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:15.509946 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcc46b8-2484-450b-b882-bedc472e857f None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:15.515215 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df414b-2b1c-4254-9606-89a0bb36da3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:15.515867 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df414b-2b1c-4254-9606-89a0bb36da3e admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:15.516148 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df414b-2b1c-4254-9606-89a0bb36da3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:15.516413 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df414b-2b1c-4254-9606-89a0bb36da3e admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:15.517175 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1df414b-2b1c-4254-9606-89a0bb36da3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:23:15.517476 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1df414b-2b1c-4254-9606-89a0bb36da3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:15.519078 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1df414b-2b1c-4254-9606-89a0bb36da3e admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:15.532341 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1347/2687] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:23:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:23:15.536872 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5079b18-e947-4abe-bd03-f48e5ff6b75a None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:15.547250 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5079b18-e947-4abe-bd03-f48e5ff6b75a None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:15.552166 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a03646-60d8-4f9a-b6d8-63081862540c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:15.552827 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a03646-60d8-4f9a-b6d8-63081862540c admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:15.553127 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a03646-60d8-4f9a-b6d8-63081862540c admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:15.553439 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a03646-60d8-4f9a-b6d8-63081862540c admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:15.554736 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a03646-60d8-4f9a-b6d8-63081862540c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:15.556646 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a03646-60d8-4f9a-b6d8-63081862540c admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:15.567605 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:15.588624 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1341/2688] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:23:15 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:23:15.592892 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df414b-2b1c-4254-9606-89a0bb36da3e admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:15.604442 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df414b-2b1c-4254-9606-89a0bb36da3e admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:15.667246 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618f07a-d720-4a1a-8d67-bf77a2afe1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:15.667949 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618f07a-d720-4a1a-8d67-bf77a2afe1ae admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:15.668230 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618f07a-d720-4a1a-8d67-bf77a2afe1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:15.668507 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618f07a-d720-4a1a-8d67-bf77a2afe1ae admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:15.669245 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3618f07a-d720-4a1a-8d67-bf77a2afe1ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:15.671204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3618f07a-d720-4a1a-8d67-bf77a2afe1ae admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:15.690003 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:15.695843 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1348/2689] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:23:15 2022] POST /identity/v3/users => generated 283 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:23:15.700306 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a03646-60d8-4f9a-b6d8-63081862540c admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:15.711980 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a03646-60d8-4f9a-b6d8-63081862540c admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:15.717718 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56051c8-9405-4a18-907c-a988752ef157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:15.718395 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56051c8-9405-4a18-907c-a988752ef157 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:15.718680 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56051c8-9405-4a18-907c-a988752ef157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:15.718953 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56051c8-9405-4a18-907c-a988752ef157 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:15.719694 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56051c8-9405-4a18-907c-a988752ef157 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:15.721342 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56051c8-9405-4a18-907c-a988752ef157 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:15.733022 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1342/2690] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:23:15.737486 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618f07a-d720-4a1a-8d67-bf77a2afe1ae admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:15.748157 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618f07a-d720-4a1a-8d67-bf77a2afe1ae admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:15.752860 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48495840-be0a-4343-9ca1-931288009ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:15.753564 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48495840-be0a-4343-9ca1-931288009ae1 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:15.753883 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48495840-be0a-4343-9ca1-931288009ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:15.754157 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48495840-be0a-4343-9ca1-931288009ae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:15.754887 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48495840-be0a-4343-9ca1-931288009ae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:15.756638 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48495840-be0a-4343-9ca1-931288009ae1 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:15.771392 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1349/2691] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:23:15.775895 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56051c8-9405-4a18-907c-a988752ef157 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:15.786623 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56051c8-9405-4a18-907c-a988752ef157 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:15.791874 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb29db-5b7b-4c2b-8440-c2e18040d43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:15.792781 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb29db-5b7b-4c2b-8440-c2e18040d43b admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:15.793096 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb29db-5b7b-4c2b-8440-c2e18040d43b admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:15.793370 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb29db-5b7b-4c2b-8440-c2e18040d43b admin admin] PATH_INFO: `/v3/projects/a50c6488882d4d38994f00a5be02b9c7/users/55bb04a375cd41d7abe02a5f8393b6c2/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:15.810333 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bb29db-5b7b-4c2b-8440-c2e18040d43b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a50c6488882d4d38994f00a5be02b9c7, user_id=55bb04a375cd41d7abe02a5f8393b6c2, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:15.812993 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bb29db-5b7b-4c2b-8440-c2e18040d43b admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:15.826873 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1343/2692] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:23:15 2022] PUT /identity/v3/projects/a50c6488882d4d38994f00a5be02b9c7/users/55bb04a375cd41d7abe02a5f8393b6c2/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:15.844359 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e60ea-5d15-40e6-8d3b-24fc4ff91f2d None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:15.844778 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e60ea-5d15-40e6-8d3b-24fc4ff91f2d None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:15.845109 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7e60ea-5d15-40e6-8d3b-24fc4ff91f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:15.876099 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-7a7e60ea-5d15-40e6-8d3b-24fc4ff91f2d None None] MFA Rules not processed for user `55bb04a375cd41d7abe02a5f8393b6c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:15.877399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-7a7e60ea-5d15-40e6-8d3b-24fc4ff91f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:15.944039 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1350/2693] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:23:15 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:15.956027 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb29db-5b7b-4c2b-8440-c2e18040d43b admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:15.958366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-82bb29db-5b7b-4c2b-8440-c2e18040d43b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:16.021640 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb29db-5b7b-4c2b-8440-c2e18040d43b admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:16.031277 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dccb5a0-4b3e-448d-9ff9-4416c8d0127c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:16.031975 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dccb5a0-4b3e-448d-9ff9-4416c8d0127c service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:16.032262 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dccb5a0-4b3e-448d-9ff9-4416c8d0127c service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:16.032539 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dccb5a0-4b3e-448d-9ff9-4416c8d0127c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:16.042863 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dccb5a0-4b3e-448d-9ff9-4416c8d0127c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:16.044305 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dccb5a0-4b3e-448d-9ff9-4416c8d0127c service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:16.055994 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1344/2694] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:15 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:23:25.940956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=37 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:26.050663 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=44,keystone:INSERT=2 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:37.216590 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebfb393-3333-483a-9233-f603b957678c None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:37.217187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebfb393-3333-483a-9233-f603b957678c None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:37.217187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebfb393-3333-483a-9233-f603b957678c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:37.240735 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-0ebfb393-3333-483a-9233-f603b957678c None None] MFA Rules not processed for user `f67110d591a94537a3df6a2d835370f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:37.243213 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-0ebfb393-3333-483a-9233-f603b957678c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:37.307087 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1351/2695] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Jun 27 21:23:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:37.364561 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dccb5a0-4b3e-448d-9ff9-4416c8d0127c service cinder] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:37.366449 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-6dccb5a0-4b3e-448d-9ff9-4416c8d0127c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:37.451807 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dccb5a0-4b3e-448d-9ff9-4416c8d0127c service cinder] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:37.457253 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516af5d1-cc40-4469-a3d8-f980b57d3bf9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bed6b231ee647d8a350da055d8e59d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:37.457942 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516af5d1-cc40-4469-a3d8-f980b57d3bf9 service swift] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:37.458208 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516af5d1-cc40-4469-a3d8-f980b57d3bf9 service swift] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:37.458488 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516af5d1-cc40-4469-a3d8-f980b57d3bf9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:37.468410 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516af5d1-cc40-4469-a3d8-f980b57d3bf9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:37.470057 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516af5d1-cc40-4469-a3d8-f980b57d3bf9 service swift] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:37.484161 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1345/2696] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Jun 27 21:23:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 21:23:47.299675 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=25 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:47.478835 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=32 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:48.542641 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebfb393-3333-483a-9233-f603b957678c None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:48.544310 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-0ebfb393-3333-483a-9233-f603b957678c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:48.640027 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebfb393-3333-483a-9233-f603b957678c None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:48.646449 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b5b6e-b57b-4d50-9b51-2f4ff6ba66c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:48.647705 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b5b6e-b57b-4d50-9b51-2f4ff6ba66c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:48.648133 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b5b6e-b57b-4d50-9b51-2f4ff6ba66c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:48.648429 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b5b6e-b57b-4d50-9b51-2f4ff6ba66c2 admin admin] PATH_INFO: `/v3/users/14dea9f30b9e4018a759ec452e19a3a6` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:48.650182 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99b5b6e-b57b-4d50-9b51-2f4ff6ba66c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14dea9f30b9e4018a759ec452e19a3a6)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:48.653107 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99b5b6e-b57b-4d50-9b51-2f4ff6ba66c2 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:48.750152 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-e99b5b6e-b57b-4d50-9b51-2f4ff6ba66c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14dea9f30b9e4018a759ec452e19a3a6'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:48.760863 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-e99b5b6e-b57b-4d50-9b51-2f4ff6ba66c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14dea9f30b9e4018a759ec452e19a3a6'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:48.761973 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-e99b5b6e-b57b-4d50-9b51-2f4ff6ba66c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14dea9f30b9e4018a759ec452e19a3a6'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:48.762768 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-e99b5b6e-b57b-4d50-9b51-2f4ff6ba66c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14dea9f30b9e4018a759ec452e19a3a6'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:48.774100 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-e99b5b6e-b57b-4d50-9b51-2f4ff6ba66c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14dea9f30b9e4018a759ec452e19a3a6'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:48.807681 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1352/2697] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:23:48 2022] DELETE /identity/v3/users/14dea9f30b9e4018a759ec452e19a3a6 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:23:48.809359 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b81e1-43de-4a9c-b163-23958cbab917 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:48.809916 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b81e1-43de-4a9c-b163-23958cbab917 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:48.809916 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b81e1-43de-4a9c-b163-23958cbab917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:48.829290 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-832b81e1-43de-4a9c-b163-23958cbab917 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:48.830800 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-832b81e1-43de-4a9c-b163-23958cbab917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:48.916965 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1346/2698] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:23:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:48.925685 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b5b6e-b57b-4d50-9b51-2f4ff6ba66c2 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:48.927343 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-e99b5b6e-b57b-4d50-9b51-2f4ff6ba66c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:49.003575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b5b6e-b57b-4d50-9b51-2f4ff6ba66c2 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:49.010377 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8214f6-3c11-45e8-b513-7f8d367b10be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:49.011111 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8214f6-3c11-45e8-b513-7f8d367b10be service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:49.011399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8214f6-3c11-45e8-b513-7f8d367b10be service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:49.011701 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8214f6-3c11-45e8-b513-7f8d367b10be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:49.023824 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8214f6-3c11-45e8-b513-7f8d367b10be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:49.025792 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8214f6-3c11-45e8-b513-7f8d367b10be service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:49.037939 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1353/2699] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:23:49.390869 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832b81e1-43de-4a9c-b163-23958cbab917 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:49.392234 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-832b81e1-43de-4a9c-b163-23958cbab917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:49.414022 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832b81e1-43de-4a9c-b163-23958cbab917 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:49.419495 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df774db-915e-48ac-9805-92eccd445055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:49.420622 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df774db-915e-48ac-9805-92eccd445055 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:49.421044 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df774db-915e-48ac-9805-92eccd445055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:49.421422 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df774db-915e-48ac-9805-92eccd445055 admin admin] PATH_INFO: `/v3/projects/c6a34e9da689495093767d8221a778a2` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:49.426954 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df774db-915e-48ac-9805-92eccd445055 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6a34e9da689495093767d8221a778a2)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:49.429613 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df774db-915e-48ac-9805-92eccd445055 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:49.504710 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-4df774db-915e-48ac-9805-92eccd445055 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6a34e9da689495093767d8221a778a2', 'request_id': 'req-4df774db-915e-48ac-9805-92eccd445055'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:49.517240 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-4df774db-915e-48ac-9805-92eccd445055 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6a34e9da689495093767d8221a778a2', 'request_id': 'req-4df774db-915e-48ac-9805-92eccd445055'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:49.529607 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-4df774db-915e-48ac-9805-92eccd445055 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6a34e9da689495093767d8221a778a2', 'request_id': 'req-4df774db-915e-48ac-9805-92eccd445055'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:49.543451 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-4df774db-915e-48ac-9805-92eccd445055 admin admin] The token cache is being invalidate because project c6a34e9da689495093767d8221a778a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:23:49.543785 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-4df774db-915e-48ac-9805-92eccd445055 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:49.544617 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-4df774db-915e-48ac-9805-92eccd445055 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:49.546567 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1347/2700] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:23:49 2022] DELETE /identity/v3/projects/c6a34e9da689495093767d8221a778a2 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:49.550890 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8214f6-3c11-45e8-b513-7f8d367b10be service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:49.562077 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8214f6-3c11-45e8-b513-7f8d367b10be service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:49.588847 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:49.655344 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16e2cac-e21b-4d4d-be8e-578de02e601c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:49.656411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e2cac-e21b-4d4d-be8e-578de02e601c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:49.656781 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e2cac-e21b-4d4d-be8e-578de02e601c admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:49.657512 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16e2cac-e21b-4d4d-be8e-578de02e601c admin admin] PATH_INFO: `/v3/users/33e6364502124e468e2ce259cce96a6e` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:49.659291 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e2cac-e21b-4d4d-be8e-578de02e601c admin admin] RBAC: Authorizing `identity:delete_user(user_id=33e6364502124e468e2ce259cce96a6e)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:49.661719 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16e2cac-e21b-4d4d-be8e-578de02e601c admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:49.740488 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-d16e2cac-e21b-4d4d-be8e-578de02e601c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33e6364502124e468e2ce259cce96a6e'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:49.751989 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-d16e2cac-e21b-4d4d-be8e-578de02e601c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33e6364502124e468e2ce259cce96a6e'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:49.752970 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-d16e2cac-e21b-4d4d-be8e-578de02e601c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33e6364502124e468e2ce259cce96a6e'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:49.753573 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-d16e2cac-e21b-4d4d-be8e-578de02e601c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33e6364502124e468e2ce259cce96a6e'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:49.762960 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-d16e2cac-e21b-4d4d-be8e-578de02e601c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33e6364502124e468e2ce259cce96a6e'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:49.808550 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1354/2701] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:23:49 2022] DELETE /identity/v3/users/33e6364502124e468e2ce259cce96a6e => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:50.206433 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df774db-915e-48ac-9805-92eccd445055 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:50.207971 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-4df774db-915e-48ac-9805-92eccd445055 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:50.232259 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df774db-915e-48ac-9805-92eccd445055 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:50.325928 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5603ad7-5fc8-46d9-9f3e-64b65d1e5ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:50.326865 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5603ad7-5fc8-46d9-9f3e-64b65d1e5ad7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:50.327184 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5603ad7-5fc8-46d9-9f3e-64b65d1e5ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:50.327483 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5603ad7-5fc8-46d9-9f3e-64b65d1e5ad7 admin admin] PATH_INFO: `/v3/projects/14daa6943ecb4bff8656ef740fe6f524` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:50.328847 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5603ad7-5fc8-46d9-9f3e-64b65d1e5ad7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14daa6943ecb4bff8656ef740fe6f524)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:50.330930 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5603ad7-5fc8-46d9-9f3e-64b65d1e5ad7 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:50.376434 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:50.400435 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-a5603ad7-5fc8-46d9-9f3e-64b65d1e5ad7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14daa6943ecb4bff8656ef740fe6f524', 'request_id': 'req-a5603ad7-5fc8-46d9-9f3e-64b65d1e5ad7'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:50.408649 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-a5603ad7-5fc8-46d9-9f3e-64b65d1e5ad7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14daa6943ecb4bff8656ef740fe6f524', 'request_id': 'req-a5603ad7-5fc8-46d9-9f3e-64b65d1e5ad7'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:50.419483 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-a5603ad7-5fc8-46d9-9f3e-64b65d1e5ad7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14daa6943ecb4bff8656ef740fe6f524', 'request_id': 'req-a5603ad7-5fc8-46d9-9f3e-64b65d1e5ad7'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:50.435317 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-a5603ad7-5fc8-46d9-9f3e-64b65d1e5ad7 admin admin] The token cache is being invalidate because project 14daa6943ecb4bff8656ef740fe6f524 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:23:50.435717 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-a5603ad7-5fc8-46d9-9f3e-64b65d1e5ad7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:50.436294 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-a5603ad7-5fc8-46d9-9f3e-64b65d1e5ad7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:50.438951 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1348/2702] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:23:50 2022] DELETE /identity/v3/projects/14daa6943ecb4bff8656ef740fe6f524 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:50.444234 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21557017-9622-4f41-9e61-bb9f024bad1a None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:50.444961 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21557017-9622-4f41-9e61-bb9f024bad1a None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:50.445249 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21557017-9622-4f41-9e61-bb9f024bad1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:50.466479 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-21557017-9622-4f41-9e61-bb9f024bad1a None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:50.467850 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-21557017-9622-4f41-9e61-bb9f024bad1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:50.556278 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1355/2703] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:23:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:50.558322 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb88c0c-bb20-49fa-890f-0b95704f37a5 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:50.558632 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb88c0c-bb20-49fa-890f-0b95704f37a5 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:50.558947 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb88c0c-bb20-49fa-890f-0b95704f37a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:50.585251 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-2cb88c0c-bb20-49fa-890f-0b95704f37a5 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:50.586818 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-2cb88c0c-bb20-49fa-890f-0b95704f37a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:50.598929 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1349/2704] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:23:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:50.603446 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21557017-9622-4f41-9e61-bb9f024bad1a None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:50.615681 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21557017-9622-4f41-9e61-bb9f024bad1a None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:50.622852 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e630aac4-8e2e-411f-8c8b-b0ac082240ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:50.623674 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e630aac4-8e2e-411f-8c8b-b0ac082240ca admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:50.624231 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e630aac4-8e2e-411f-8c8b-b0ac082240ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:50.624670 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e630aac4-8e2e-411f-8c8b-b0ac082240ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:50.626182 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e630aac4-8e2e-411f-8c8b-b0ac082240ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:23:50.626931 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e630aac4-8e2e-411f-8c8b-b0ac082240ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:50.629572 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e630aac4-8e2e-411f-8c8b-b0ac082240ca admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:50.648049 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1356/2705] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:23:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:23:50.649421 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb88c0c-bb20-49fa-890f-0b95704f37a5 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:50.659625 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb88c0c-bb20-49fa-890f-0b95704f37a5 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:50.665650 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa682a-27cb-4bb2-8161-67d377bff92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:50.666410 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa682a-27cb-4bb2-8161-67d377bff92b admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:50.666695 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa682a-27cb-4bb2-8161-67d377bff92b admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:50.666994 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa682a-27cb-4bb2-8161-67d377bff92b admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:50.667745 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aa682a-27cb-4bb2-8161-67d377bff92b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:50.669962 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aa682a-27cb-4bb2-8161-67d377bff92b admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:50.714518 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1350/2706] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:23:50 2022] POST /identity/v3/projects => generated 365 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:23:50.715859 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e630aac4-8e2e-411f-8c8b-b0ac082240ca admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:50.727906 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e630aac4-8e2e-411f-8c8b-b0ac082240ca admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:50.763558 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:50.809821 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ec2a00-9135-488e-a8dd-0ca674d1eeaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:50.810552 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec2a00-9135-488e-a8dd-0ca674d1eeaf admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:50.810846 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec2a00-9135-488e-a8dd-0ca674d1eeaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:50.811119 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec2a00-9135-488e-a8dd-0ca674d1eeaf admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:50.811986 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ec2a00-9135-488e-a8dd-0ca674d1eeaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:50.814123 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ec2a00-9135-488e-a8dd-0ca674d1eeaf admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:50.844176 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1357/2707] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:23:50 2022] POST /identity/v3/users => generated 283 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:23:50.848577 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa682a-27cb-4bb2-8161-67d377bff92b admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:50.864088 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa682a-27cb-4bb2-8161-67d377bff92b admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:50.870841 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba961d6-9c3c-46d2-805c-637c63f4ca83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:50.871694 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba961d6-9c3c-46d2-805c-637c63f4ca83 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:50.871998 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba961d6-9c3c-46d2-805c-637c63f4ca83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:50.872241 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba961d6-9c3c-46d2-805c-637c63f4ca83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:50.873099 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba961d6-9c3c-46d2-805c-637c63f4ca83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:50.875068 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba961d6-9c3c-46d2-805c-637c63f4ca83 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:50.888582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1351/2708] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:50 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:23:50.893126 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ec2a00-9135-488e-a8dd-0ca674d1eeaf admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:50.905641 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ec2a00-9135-488e-a8dd-0ca674d1eeaf admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:50.912150 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf0918-5e12-42b6-be05-63ec464e51f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:50.913084 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf0918-5e12-42b6-be05-63ec464e51f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:50.913475 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf0918-5e12-42b6-be05-63ec464e51f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:50.914150 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf0918-5e12-42b6-be05-63ec464e51f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:50.915188 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cf0918-5e12-42b6-be05-63ec464e51f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:50.917402 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cf0918-5e12-42b6-be05-63ec464e51f2 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:50.936974 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1358/2709] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:50 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:23:50.940657 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba961d6-9c3c-46d2-805c-637c63f4ca83 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:50.955166 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba961d6-9c3c-46d2-805c-637c63f4ca83 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:50.962437 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6d1f4-8128-4c2f-a8bd-d74cc965bfaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:50.963640 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6d1f4-8128-4c2f-a8bd-d74cc965bfaa admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:50.964165 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6d1f4-8128-4c2f-a8bd-d74cc965bfaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:50.964875 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6d1f4-8128-4c2f-a8bd-d74cc965bfaa admin admin] PATH_INFO: `/v3/projects/6ed075f1bd4d4137bed07f49f5b72a87/users/02df3fe1d2664f6bbcd334afeabad75f/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:50.986108 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f6d1f4-8128-4c2f-a8bd-d74cc965bfaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ed075f1bd4d4137bed07f49f5b72a87, user_id=02df3fe1d2664f6bbcd334afeabad75f, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:50.988727 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f6d1f4-8128-4c2f-a8bd-d74cc965bfaa admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:51.007286 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1352/2710] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:23:50 2022] PUT /identity/v3/projects/6ed075f1bd4d4137bed07f49f5b72a87/users/02df3fe1d2664f6bbcd334afeabad75f/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:23:51.020952 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf6781-6811-4981-9b3e-dae9923cdfab None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:51.021317 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf6781-6811-4981-9b3e-dae9923cdfab None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:51.021698 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf6781-6811-4981-9b3e-dae9923cdfab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:51.075533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-a8bf6781-6811-4981-9b3e-dae9923cdfab None None] MFA Rules not processed for user `02df3fe1d2664f6bbcd334afeabad75f`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:51.077719 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a8bf6781-6811-4981-9b3e-dae9923cdfab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:51.173134 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1359/2711] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:23:51 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:23:51.174568 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6d1f4-8128-4c2f-a8bd-d74cc965bfaa admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:51.255764 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6d1f4-8128-4c2f-a8bd-d74cc965bfaa admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:51.262029 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac3798d-7cde-42bb-8e8d-269d4b0f6c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:51.262852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac3798d-7cde-42bb-8e8d-269d4b0f6c33 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:51.263328 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac3798d-7cde-42bb-8e8d-269d4b0f6c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:51.263586 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac3798d-7cde-42bb-8e8d-269d4b0f6c33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:51.264348 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac3798d-7cde-42bb-8e8d-269d4b0f6c33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:51.266449 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac3798d-7cde-42bb-8e8d-269d4b0f6c33 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:51.301172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bf6781-6811-4981-9b3e-dae9923cdfab None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:51.303546 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1353/2712] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:23:51 2022] POST /identity/v3/projects => generated 365 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:23:51.319996 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bf6781-6811-4981-9b3e-dae9923cdfab None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:51.402502 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705370cb-5e62-4ece-ac8c-519cef43a308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:51.403428 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705370cb-5e62-4ece-ac8c-519cef43a308 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:51.403913 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705370cb-5e62-4ece-ac8c-519cef43a308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:51.404249 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705370cb-5e62-4ece-ac8c-519cef43a308 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:51.405134 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705370cb-5e62-4ece-ac8c-519cef43a308 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:51.407298 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705370cb-5e62-4ece-ac8c-519cef43a308 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:51.457606 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1360/2713] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:23:51 2022] POST /identity/v3/users => generated 289 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:23:51.462453 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac3798d-7cde-42bb-8e8d-269d4b0f6c33 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:51.478835 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac3798d-7cde-42bb-8e8d-269d4b0f6c33 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:51.486612 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdef560-a566-4ca5-a9e1-f5edaf0e5d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:51.487605 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdef560-a566-4ca5-a9e1-f5edaf0e5d0d admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:51.488101 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdef560-a566-4ca5-a9e1-f5edaf0e5d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:51.488570 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdef560-a566-4ca5-a9e1-f5edaf0e5d0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:51.489190 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdef560-a566-4ca5-a9e1-f5edaf0e5d0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:51.491112 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdef560-a566-4ca5-a9e1-f5edaf0e5d0d admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:51.507865 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1354/2714] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:51 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:23:51.511784 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705370cb-5e62-4ece-ac8c-519cef43a308 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:51.525285 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705370cb-5e62-4ece-ac8c-519cef43a308 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:51.532341 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69067f42-d11d-4f24-8f2e-c7808f1b32a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:51.533628 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69067f42-d11d-4f24-8f2e-c7808f1b32a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:51.534122 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69067f42-d11d-4f24-8f2e-c7808f1b32a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:51.534328 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69067f42-d11d-4f24-8f2e-c7808f1b32a8 admin admin] PATH_INFO: `/v3/projects/759e6d926b5e4c829267da91e0ec70d3/users/beb46f61b9ef465c913e10f1c9aed7d9/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:51.584328 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69067f42-d11d-4f24-8f2e-c7808f1b32a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=759e6d926b5e4c829267da91e0ec70d3, user_id=beb46f61b9ef465c913e10f1c9aed7d9, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:51.592047 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69067f42-d11d-4f24-8f2e-c7808f1b32a8 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:51.643692 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1361/2715] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:23:51 2022] PUT /identity/v3/projects/759e6d926b5e4c829267da91e0ec70d3/users/beb46f61b9ef465c913e10f1c9aed7d9/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:51.648013 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdef560-a566-4ca5-a9e1-f5edaf0e5d0d admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:51.663657 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdef560-a566-4ca5-a9e1-f5edaf0e5d0d admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:51.690214 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:51.773805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0ba237-8cb9-48ce-9c42-aebcae0b1a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:51.774732 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ba237-8cb9-48ce-9c42-aebcae0b1a90 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:51.775125 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ba237-8cb9-48ce-9c42-aebcae0b1a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:51.775613 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ba237-8cb9-48ce-9c42-aebcae0b1a90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:51.776913 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0ba237-8cb9-48ce-9c42-aebcae0b1a90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:51.780223 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0ba237-8cb9-48ce-9c42-aebcae0b1a90 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:51.799897 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1355/2716] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:51 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:23:51.801185 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69067f42-d11d-4f24-8f2e-c7808f1b32a8 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:51.808982 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:51.816963 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69067f42-d11d-4f24-8f2e-c7808f1b32a8 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:51.827990 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98236330-75b8-4a82-acde-858e96826307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:51.829988 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98236330-75b8-4a82-acde-858e96826307 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:51.830453 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98236330-75b8-4a82-acde-858e96826307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:51.830754 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98236330-75b8-4a82-acde-858e96826307 admin admin] PATH_INFO: `/v3/projects/759e6d926b5e4c829267da91e0ec70d3/users/beb46f61b9ef465c913e10f1c9aed7d9/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:51.834054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98236330-75b8-4a82-acde-858e96826307 admin admin] RBAC: Authorizing `identity:create_grant(project_id=759e6d926b5e4c829267da91e0ec70d3, user_id=beb46f61b9ef465c913e10f1c9aed7d9, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:51.838081 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98236330-75b8-4a82-acde-858e96826307 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:51.854783 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1362/2717] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:23:51 2022] PUT /identity/v3/projects/759e6d926b5e4c829267da91e0ec70d3/users/beb46f61b9ef465c913e10f1c9aed7d9/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:23:51.867331 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc2896d-0231-4f16-98e4-0d2050833a98 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:51.867617 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc2896d-0231-4f16-98e4-0d2050833a98 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:51.867912 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc2896d-0231-4f16-98e4-0d2050833a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:51.925369 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-fbc2896d-0231-4f16-98e4-0d2050833a98 None None] MFA Rules not processed for user `beb46f61b9ef465c913e10f1c9aed7d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:51.929518 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-fbc2896d-0231-4f16-98e4-0d2050833a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:52.062533 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1356/2718] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:23:51 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:52.078290 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98236330-75b8-4a82-acde-858e96826307 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:52.080270 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-98236330-75b8-4a82-acde-858e96826307 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:52.159604 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98236330-75b8-4a82-acde-858e96826307 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:52.165312 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e56422-6840-4c86-912e-50013273ddd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:52.166529 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e56422-6840-4c86-912e-50013273ddd6 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:52.167651 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e56422-6840-4c86-912e-50013273ddd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:52.168098 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e56422-6840-4c86-912e-50013273ddd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:52.179290 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e56422-6840-4c86-912e-50013273ddd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:52.184155 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e56422-6840-4c86-912e-50013273ddd6 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:52.204368 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1363/2719] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:52 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:23:52.728555 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc2896d-0231-4f16-98e4-0d2050833a98 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:52.811258 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc2896d-0231-4f16-98e4-0d2050833a98 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:52.816621 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e0c250-372b-4e84-b4ed-0d63254a9a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:52.817589 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0c250-372b-4e84-b4ed-0d63254a9a17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:52.818073 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0c250-372b-4e84-b4ed-0d63254a9a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:52.818307 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e0c250-372b-4e84-b4ed-0d63254a9a17 admin admin] PATH_INFO: `/v3/users/02df3fe1d2664f6bbcd334afeabad75f` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:52.820123 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e0c250-372b-4e84-b4ed-0d63254a9a17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02df3fe1d2664f6bbcd334afeabad75f)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:52.822187 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e0c250-372b-4e84-b4ed-0d63254a9a17 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:52.886556 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-d5e0c250-372b-4e84-b4ed-0d63254a9a17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02df3fe1d2664f6bbcd334afeabad75f'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:52.889980 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:52.898568 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-d5e0c250-372b-4e84-b4ed-0d63254a9a17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02df3fe1d2664f6bbcd334afeabad75f'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:52.899602 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-d5e0c250-372b-4e84-b4ed-0d63254a9a17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02df3fe1d2664f6bbcd334afeabad75f'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:52.900269 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-d5e0c250-372b-4e84-b4ed-0d63254a9a17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02df3fe1d2664f6bbcd334afeabad75f'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:52.910225 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-d5e0c250-372b-4e84-b4ed-0d63254a9a17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02df3fe1d2664f6bbcd334afeabad75f'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:52.932252 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1357/2720] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:23:52 2022] DELETE /identity/v3/users/02df3fe1d2664f6bbcd334afeabad75f => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:52.937339 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7078d-c7e0-47a5-ad8a-ef413e337342 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:52.938094 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7078d-c7e0-47a5-ad8a-ef413e337342 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:52.938094 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7078d-c7e0-47a5-ad8a-ef413e337342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:52.961283 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-f8f7078d-c7e0-47a5-ad8a-ef413e337342 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:52.962553 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-f8f7078d-c7e0-47a5-ad8a-ef413e337342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:53.048942 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1364/2721] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:23:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:53.063615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e0c250-372b-4e84-b4ed-0d63254a9a17 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:53.065416 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-d5e0c250-372b-4e84-b4ed-0d63254a9a17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:53.146111 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e0c250-372b-4e84-b4ed-0d63254a9a17 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:53.152305 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5014924-811b-4348-8483-5ad2483070a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:53.153078 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5014924-811b-4348-8483-5ad2483070a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:53.153373 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5014924-811b-4348-8483-5ad2483070a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:53.153819 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5014924-811b-4348-8483-5ad2483070a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:53.170228 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5014924-811b-4348-8483-5ad2483070a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:53.171683 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5014924-811b-4348-8483-5ad2483070a7 service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:53.186421 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1358/2722] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:23:53.558755 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f7078d-c7e0-47a5-ad8a-ef413e337342 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:53.561603 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-f8f7078d-c7e0-47a5-ad8a-ef413e337342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:53.612942 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f7078d-c7e0-47a5-ad8a-ef413e337342 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:53.622047 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5a12e6-bfeb-4725-b895-3ba11cc20fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:53.623549 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a12e6-bfeb-4725-b895-3ba11cc20fd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:53.623903 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a12e6-bfeb-4725-b895-3ba11cc20fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:53.624485 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a12e6-bfeb-4725-b895-3ba11cc20fd9 admin admin] PATH_INFO: `/v3/projects/6ed075f1bd4d4137bed07f49f5b72a87` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:53.626321 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5a12e6-bfeb-4725-b895-3ba11cc20fd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ed075f1bd4d4137bed07f49f5b72a87)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:53.629552 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5a12e6-bfeb-4725-b895-3ba11cc20fd9 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:53.698233 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8d5a12e6-bfeb-4725-b895-3ba11cc20fd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ed075f1bd4d4137bed07f49f5b72a87', 'request_id': 'req-8d5a12e6-bfeb-4725-b895-3ba11cc20fd9'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:53.706883 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8d5a12e6-bfeb-4725-b895-3ba11cc20fd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ed075f1bd4d4137bed07f49f5b72a87', 'request_id': 'req-8d5a12e6-bfeb-4725-b895-3ba11cc20fd9'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:53.721483 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8d5a12e6-bfeb-4725-b895-3ba11cc20fd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ed075f1bd4d4137bed07f49f5b72a87', 'request_id': 'req-8d5a12e6-bfeb-4725-b895-3ba11cc20fd9'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:53.746223 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8d5a12e6-bfeb-4725-b895-3ba11cc20fd9 admin admin] The token cache is being invalidate because project 6ed075f1bd4d4137bed07f49f5b72a87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:23:53.746863 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8d5a12e6-bfeb-4725-b895-3ba11cc20fd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:53.748060 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8d5a12e6-bfeb-4725-b895-3ba11cc20fd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:53.750455 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1365/2723] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:23:53 2022] DELETE /identity/v3/projects/6ed075f1bd4d4137bed07f49f5b72a87 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:53.755083 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5014924-811b-4348-8483-5ad2483070a7 service neutron] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:53.771374 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5014924-811b-4348-8483-5ad2483070a7 service neutron] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:53.872600 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712511f1-d9df-4012-89a1-22a269f24563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:53.873502 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712511f1-d9df-4012-89a1-22a269f24563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:53.873687 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712511f1-d9df-4012-89a1-22a269f24563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:53.873967 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712511f1-d9df-4012-89a1-22a269f24563 admin admin] PATH_INFO: `/v3/users/beb46f61b9ef465c913e10f1c9aed7d9` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:53.875615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712511f1-d9df-4012-89a1-22a269f24563 admin admin] RBAC: Authorizing `identity:delete_user(user_id=beb46f61b9ef465c913e10f1c9aed7d9)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:53.878000 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712511f1-d9df-4012-89a1-22a269f24563 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:53.956846 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-712511f1-d9df-4012-89a1-22a269f24563 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'beb46f61b9ef465c913e10f1c9aed7d9'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:53.967822 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-712511f1-d9df-4012-89a1-22a269f24563 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'beb46f61b9ef465c913e10f1c9aed7d9'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:53.968669 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-712511f1-d9df-4012-89a1-22a269f24563 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'beb46f61b9ef465c913e10f1c9aed7d9'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:53.969409 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-712511f1-d9df-4012-89a1-22a269f24563 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'beb46f61b9ef465c913e10f1c9aed7d9'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:53.980714 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-712511f1-d9df-4012-89a1-22a269f24563 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'beb46f61b9ef465c913e10f1c9aed7d9'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:53.988810 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:54.008445 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1359/2724] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:23:53 2022] DELETE /identity/v3/users/beb46f61b9ef465c913e10f1c9aed7d9 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:54.375077 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5a12e6-bfeb-4725-b895-3ba11cc20fd9 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:54.378009 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-8d5a12e6-bfeb-4725-b895-3ba11cc20fd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:54.401420 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5a12e6-bfeb-4725-b895-3ba11cc20fd9 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:54.423409 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:54.500583 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c8e76-64d2-4ca3-8cf0-317e568c77f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:54.502114 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c8e76-64d2-4ca3-8cf0-317e568c77f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:54.502397 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c8e76-64d2-4ca3-8cf0-317e568c77f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:54.502680 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c8e76-64d2-4ca3-8cf0-317e568c77f4 admin admin] PATH_INFO: `/v3/projects/759e6d926b5e4c829267da91e0ec70d3` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:54.504109 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c8e76-64d2-4ca3-8cf0-317e568c77f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=759e6d926b5e4c829267da91e0ec70d3)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:54.506855 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c8e76-64d2-4ca3-8cf0-317e568c77f4 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:54.564256 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-d31c8e76-64d2-4ca3-8cf0-317e568c77f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '759e6d926b5e4c829267da91e0ec70d3', 'request_id': 'req-d31c8e76-64d2-4ca3-8cf0-317e568c77f4'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:54.571121 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-d31c8e76-64d2-4ca3-8cf0-317e568c77f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '759e6d926b5e4c829267da91e0ec70d3', 'request_id': 'req-d31c8e76-64d2-4ca3-8cf0-317e568c77f4'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:54.579796 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-d31c8e76-64d2-4ca3-8cf0-317e568c77f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '759e6d926b5e4c829267da91e0ec70d3', 'request_id': 'req-d31c8e76-64d2-4ca3-8cf0-317e568c77f4'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:54.592232 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-d31c8e76-64d2-4ca3-8cf0-317e568c77f4 admin admin] The token cache is being invalidate because project 759e6d926b5e4c829267da91e0ec70d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:23:54.592232 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-d31c8e76-64d2-4ca3-8cf0-317e568c77f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:54.593119 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-d31c8e76-64d2-4ca3-8cf0-317e568c77f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:23:54.594988 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1366/2725] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:23:54 2022] DELETE /identity/v3/projects/759e6d926b5e4c829267da91e0ec70d3 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:54.601062 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb186bc-6135-4b56-9967-392b9fc02f59 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:54.601341 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb186bc-6135-4b56-9967-392b9fc02f59 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:54.601627 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb186bc-6135-4b56-9967-392b9fc02f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:54.621400 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-aeb186bc-6135-4b56-9967-392b9fc02f59 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:54.622708 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-aeb186bc-6135-4b56-9967-392b9fc02f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:54.700261 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1360/2726] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:23:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:54.710699 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3bdc12-99be-4aa2-8ca7-2e401f647c4f None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:54.711568 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3bdc12-99be-4aa2-8ca7-2e401f647c4f None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:54.712111 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3bdc12-99be-4aa2-8ca7-2e401f647c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:54.735760 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-9a3bdc12-99be-4aa2-8ca7-2e401f647c4f None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:54.737175 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-9a3bdc12-99be-4aa2-8ca7-2e401f647c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:54.746702 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1367/2727] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:23:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:54.752400 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb186bc-6135-4b56-9967-392b9fc02f59 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:54.767737 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb186bc-6135-4b56-9967-392b9fc02f59 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:54.773730 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da52f14-bd97-43ff-b813-9b7988f3a1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:54.774458 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da52f14-bd97-43ff-b813-9b7988f3a1a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:54.774754 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da52f14-bd97-43ff-b813-9b7988f3a1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:54.775073 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da52f14-bd97-43ff-b813-9b7988f3a1a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:54.775869 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da52f14-bd97-43ff-b813-9b7988f3a1a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:23:54.776201 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da52f14-bd97-43ff-b813-9b7988f3a1a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:54.777947 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da52f14-bd97-43ff-b813-9b7988f3a1a2 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:54.790934 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1361/2728] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:23:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:23:54.795575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3bdc12-99be-4aa2-8ca7-2e401f647c4f None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:54.805514 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3bdc12-99be-4aa2-8ca7-2e401f647c4f None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:54.819798 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626aadc6-7aa0-486e-becd-3b7e95ee1eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:54.820574 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626aadc6-7aa0-486e-becd-3b7e95ee1eea admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:54.821533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626aadc6-7aa0-486e-becd-3b7e95ee1eea admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:54.821533 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626aadc6-7aa0-486e-becd-3b7e95ee1eea admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:54.822445 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626aadc6-7aa0-486e-becd-3b7e95ee1eea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:54.824777 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626aadc6-7aa0-486e-becd-3b7e95ee1eea admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:54.869493 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1368/2729] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:23:54 2022] POST /identity/v3/projects => generated 363 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:23:54.871097 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da52f14-bd97-43ff-b813-9b7988f3a1a2 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:54.888686 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da52f14-bd97-43ff-b813-9b7988f3a1a2 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:54.970404 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc1631b-3fad-4e95-92ef-43ba9dcf3f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:54.970998 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc1631b-3fad-4e95-92ef-43ba9dcf3f2e admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:54.971282 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc1631b-3fad-4e95-92ef-43ba9dcf3f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:54.971559 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc1631b-3fad-4e95-92ef-43ba9dcf3f2e admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:54.972274 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc1631b-3fad-4e95-92ef-43ba9dcf3f2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:54.974260 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc1631b-3fad-4e95-92ef-43ba9dcf3f2e admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:54.997286 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1362/2730] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:23:54 2022] POST /identity/v3/users => generated 282 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:23:55.001215 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626aadc6-7aa0-486e-becd-3b7e95ee1eea admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:55.017272 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626aadc6-7aa0-486e-becd-3b7e95ee1eea admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:55.017652 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ecf479-7ab4-415a-89a9-c297ee82116e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:55.018282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ecf479-7ab4-415a-89a9-c297ee82116e admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:55.018567 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ecf479-7ab4-415a-89a9-c297ee82116e admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:55.018930 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ecf479-7ab4-415a-89a9-c297ee82116e admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:55.019869 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ecf479-7ab4-415a-89a9-c297ee82116e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:55.021795 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ecf479-7ab4-415a-89a9-c297ee82116e admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:55.046434 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1369/2731] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:23:55.050824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc1631b-3fad-4e95-92ef-43ba9dcf3f2e admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:55.062830 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc1631b-3fad-4e95-92ef-43ba9dcf3f2e admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:55.067599 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd9173c-38b0-4712-9e60-66d63043ff59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:55.068272 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9173c-38b0-4712-9e60-66d63043ff59 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:55.068632 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9173c-38b0-4712-9e60-66d63043ff59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:55.068845 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9173c-38b0-4712-9e60-66d63043ff59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:55.069618 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd9173c-38b0-4712-9e60-66d63043ff59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:55.071222 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd9173c-38b0-4712-9e60-66d63043ff59 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:55.082841 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1363/2732] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:23:55.087208 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ecf479-7ab4-415a-89a9-c297ee82116e admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:55.097904 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ecf479-7ab4-415a-89a9-c297ee82116e admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:55.104867 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5b90d8-f916-40e8-a41d-88ae810dc8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:55.105812 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b90d8-f916-40e8-a41d-88ae810dc8d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:55.106096 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b90d8-f916-40e8-a41d-88ae810dc8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:55.106563 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b90d8-f916-40e8-a41d-88ae810dc8d3 admin admin] PATH_INFO: `/v3/projects/15a8108ef16e4046a829ac762b04510b/users/ac486c06b38742198fc56a855d70aa18/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:55.122208 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5b90d8-f916-40e8-a41d-88ae810dc8d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15a8108ef16e4046a829ac762b04510b, user_id=ac486c06b38742198fc56a855d70aa18, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:55.125048 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5b90d8-f916-40e8-a41d-88ae810dc8d3 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:55.142993 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1370/2733] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:23:55 2022] PUT /identity/v3/projects/15a8108ef16e4046a829ac762b04510b/users/ac486c06b38742198fc56a855d70aa18/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:55.155232 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59909f45-5f68-4bbd-b4e8-874780c2a252 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:55.155947 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59909f45-5f68-4bbd-b4e8-874780c2a252 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:55.156262 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59909f45-5f68-4bbd-b4e8-874780c2a252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:55.187111 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-59909f45-5f68-4bbd-b4e8-874780c2a252 None None] MFA Rules not processed for user `ac486c06b38742198fc56a855d70aa18`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:55.188257 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-59909f45-5f68-4bbd-b4e8-874780c2a252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:55.233262 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:55.260960 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1364/2734] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:23:55 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:55.265860 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5b90d8-f916-40e8-a41d-88ae810dc8d3 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:55.340032 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:55.384640 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5b90d8-f916-40e8-a41d-88ae810dc8d3 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:55.402221 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447c7004-745b-47d9-abe1-ba8d52c53dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:55.403452 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c7004-745b-47d9-abe1-ba8d52c53dd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:55.404261 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c7004-745b-47d9-abe1-ba8d52c53dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:55.404804 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c7004-745b-47d9-abe1-ba8d52c53dd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:55.406180 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447c7004-745b-47d9-abe1-ba8d52c53dd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:55.409873 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447c7004-745b-47d9-abe1-ba8d52c53dd1 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:55.464007 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1371/2735] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:23:55 2022] POST /identity/v3/projects => generated 363 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:23:55.469051 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59909f45-5f68-4bbd-b4e8-874780c2a252 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:55.483780 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59909f45-5f68-4bbd-b4e8-874780c2a252 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:55.573955 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615625a-0f95-4082-813c-24ab6878553a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:55.575012 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615625a-0f95-4082-813c-24ab6878553a admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:55.575382 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615625a-0f95-4082-813c-24ab6878553a admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:55.575652 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615625a-0f95-4082-813c-24ab6878553a admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:55.576792 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2615625a-0f95-4082-813c-24ab6878553a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:55.580176 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2615625a-0f95-4082-813c-24ab6878553a admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:55.618496 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1365/2736] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:23:55 2022] POST /identity/v3/users => generated 288 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:23:55.618685 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447c7004-745b-47d9-abe1-ba8d52c53dd1 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:55.632501 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447c7004-745b-47d9-abe1-ba8d52c53dd1 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:55.639778 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72efa48b-f7a0-4a7f-92a9-eebaa8c9af51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:55.641144 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72efa48b-f7a0-4a7f-92a9-eebaa8c9af51 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:55.641633 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72efa48b-f7a0-4a7f-92a9-eebaa8c9af51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:55.642160 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72efa48b-f7a0-4a7f-92a9-eebaa8c9af51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:55.643058 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72efa48b-f7a0-4a7f-92a9-eebaa8c9af51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:55.644905 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72efa48b-f7a0-4a7f-92a9-eebaa8c9af51 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:55.660974 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1372/2737] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:23:55.666227 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615625a-0f95-4082-813c-24ab6878553a admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:55.679901 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615625a-0f95-4082-813c-24ab6878553a admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:55.687046 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424aa0ea-9df8-47f0-82e5-16a838a2fffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:55.688280 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424aa0ea-9df8-47f0-82e5-16a838a2fffe admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:55.688676 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424aa0ea-9df8-47f0-82e5-16a838a2fffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:55.689132 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424aa0ea-9df8-47f0-82e5-16a838a2fffe admin admin] PATH_INFO: `/v3/projects/aa905a32ff4b49679051eec9159fd91e/users/c022de91e2984dcb8370d2cfe506a799/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:55.723446 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424aa0ea-9df8-47f0-82e5-16a838a2fffe admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa905a32ff4b49679051eec9159fd91e, user_id=c022de91e2984dcb8370d2cfe506a799, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:55.729340 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424aa0ea-9df8-47f0-82e5-16a838a2fffe admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:55.748686 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1366/2738] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:23:55 2022] PUT /identity/v3/projects/aa905a32ff4b49679051eec9159fd91e/users/c022de91e2984dcb8370d2cfe506a799/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:23:55.752854 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72efa48b-f7a0-4a7f-92a9-eebaa8c9af51 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:55.769061 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72efa48b-f7a0-4a7f-92a9-eebaa8c9af51 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:55.839383 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9194d5d-74a8-4a46-9372-c34ef2099702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:55.840313 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9194d5d-74a8-4a46-9372-c34ef2099702 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:55.840651 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9194d5d-74a8-4a46-9372-c34ef2099702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:55.841017 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9194d5d-74a8-4a46-9372-c34ef2099702 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:55.841826 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9194d5d-74a8-4a46-9372-c34ef2099702 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:55.843793 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9194d5d-74a8-4a46-9372-c34ef2099702 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:55.857114 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1373/2739] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:23:55.861492 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424aa0ea-9df8-47f0-82e5-16a838a2fffe admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:55.874321 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424aa0ea-9df8-47f0-82e5-16a838a2fffe admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:55.884860 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6051ccf-caf6-48f0-9dca-3d11691a4272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:55.886059 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6051ccf-caf6-48f0-9dca-3d11691a4272 admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:55.886234 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6051ccf-caf6-48f0-9dca-3d11691a4272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:55.886642 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6051ccf-caf6-48f0-9dca-3d11691a4272 admin admin] PATH_INFO: `/v3/projects/aa905a32ff4b49679051eec9159fd91e/users/c022de91e2984dcb8370d2cfe506a799/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:55.889018 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6051ccf-caf6-48f0-9dca-3d11691a4272 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa905a32ff4b49679051eec9159fd91e, user_id=c022de91e2984dcb8370d2cfe506a799, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:55.891955 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6051ccf-caf6-48f0-9dca-3d11691a4272 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:55.912047 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1367/2740] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:23:55 2022] PUT /identity/v3/projects/aa905a32ff4b49679051eec9159fd91e/users/c022de91e2984dcb8370d2cfe506a799/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:23:55.924876 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd572030-1157-4add-8321-4920a1aee173 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:55.925288 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd572030-1157-4add-8321-4920a1aee173 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:55.925666 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd572030-1157-4add-8321-4920a1aee173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:55.965269 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-fd572030-1157-4add-8321-4920a1aee173 None None] MFA Rules not processed for user `c022de91e2984dcb8370d2cfe506a799`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:55.966598 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-fd572030-1157-4add-8321-4920a1aee173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:56.080323 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1374/2741] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:23:55 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:56.092832 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6051ccf-caf6-48f0-9dca-3d11691a4272 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:56.094729 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-e6051ccf-caf6-48f0-9dca-3d11691a4272 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:56.219863 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6051ccf-caf6-48f0-9dca-3d11691a4272 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:56.223824 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01ea993-a406-4e44-aaf8-a548df55249b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:56.224497 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ea993-a406-4e44-aaf8-a548df55249b service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:56.224843 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ea993-a406-4e44-aaf8-a548df55249b service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:56.225154 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ea993-a406-4e44-aaf8-a548df55249b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:56.240622 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01ea993-a406-4e44-aaf8-a548df55249b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:56.242223 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01ea993-a406-4e44-aaf8-a548df55249b service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:56.257062 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1368/2742] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:56 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:23:56.338335 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd572030-1157-4add-8321-4920a1aee173 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:56.352351 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd572030-1157-4add-8321-4920a1aee173 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:56.357739 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b93524-0228-43be-956b-767510da3cb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:56.358483 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b93524-0228-43be-956b-767510da3cb4 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:56.358771 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b93524-0228-43be-956b-767510da3cb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:56.359048 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b93524-0228-43be-956b-767510da3cb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:56.403740 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:56.433077 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b93524-0228-43be-956b-767510da3cb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:56.434953 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b93524-0228-43be-956b-767510da3cb4 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:56.448973 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1375/2743] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:23:56 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:23:59.187301 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25debfd9-34ba-4ddb-b0b5-954147edaad2 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:59.187911 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25debfd9-34ba-4ddb-b0b5-954147edaad2 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:59.187911 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25debfd9-34ba-4ddb-b0b5-954147edaad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:59.207085 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-25debfd9-34ba-4ddb-b0b5-954147edaad2 None None] MFA Rules not processed for user `f67110d591a94537a3df6a2d835370f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:23:59.208322 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-25debfd9-34ba-4ddb-b0b5-954147edaad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:59.243020 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:23:59.271296 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1369/2744] 10.222.0.13 () {58 vars in 967 bytes} [Mon Jun 27 21:23:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:23:59.291783 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b93524-0228-43be-956b-767510da3cb4 service cinder] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:23:59.292618 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-97b93524-0228-43be-956b-767510da3cb4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:23:59.376206 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b93524-0228-43be-956b-767510da3cb4 service cinder] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:23:59.381682 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487dc128-00cc-4d54-8e0f-5f01c7997088 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bed6b231ee647d8a350da055d8e59d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:23:59.382352 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487dc128-00cc-4d54-8e0f-5f01c7997088 service swift] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:23:59.382632 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487dc128-00cc-4d54-8e0f-5f01c7997088 service swift] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:23:59.382902 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487dc128-00cc-4d54-8e0f-5f01c7997088 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:23:59.392837 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487dc128-00cc-4d54-8e0f-5f01c7997088 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:23:59.394689 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487dc128-00cc-4d54-8e0f-5f01c7997088 service swift] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:23:59.405569 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1376/2745] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Jun 27 21:23:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 21:24:03.587401 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25debfd9-34ba-4ddb-b0b5-954147edaad2 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:03.589717 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-25debfd9-34ba-4ddb-b0b5-954147edaad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:03.671211 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25debfd9-34ba-4ddb-b0b5-954147edaad2 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:03.676871 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c79c0-5e97-46f7-adc8-0b8661020906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:03.677849 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c79c0-5e97-46f7-adc8-0b8661020906 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:03.678134 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c79c0-5e97-46f7-adc8-0b8661020906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:03.678409 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c79c0-5e97-46f7-adc8-0b8661020906 admin admin] PATH_INFO: `/v3/users/e0e171895ccc4f1cbc69ead21175fc5d` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:03.679811 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62c79c0-5e97-46f7-adc8-0b8661020906 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0e171895ccc4f1cbc69ead21175fc5d)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:03.681883 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62c79c0-5e97-46f7-adc8-0b8661020906 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:03.731176 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-b62c79c0-5e97-46f7-adc8-0b8661020906 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0e171895ccc4f1cbc69ead21175fc5d'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:03.737981 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-b62c79c0-5e97-46f7-adc8-0b8661020906 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0e171895ccc4f1cbc69ead21175fc5d'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:03.738739 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-b62c79c0-5e97-46f7-adc8-0b8661020906 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0e171895ccc4f1cbc69ead21175fc5d'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:03.739365 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-b62c79c0-5e97-46f7-adc8-0b8661020906 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0e171895ccc4f1cbc69ead21175fc5d'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:03.747740 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-b62c79c0-5e97-46f7-adc8-0b8661020906 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0e171895ccc4f1cbc69ead21175fc5d'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:03.766338 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1370/2746] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:24:03 2022] DELETE /identity/v3/users/e0e171895ccc4f1cbc69ead21175fc5d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:24:03.956545 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487dc128-00cc-4d54-8e0f-5f01c7997088 service swift] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:03.972165 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487dc128-00cc-4d54-8e0f-5f01c7997088 service swift] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:04.041679 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c05f2-e640-4ba4-8246-293cec7a53bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:04.043000 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c05f2-e640-4ba4-8246-293cec7a53bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:04.043241 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c05f2-e640-4ba4-8246-293cec7a53bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:04.043530 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c05f2-e640-4ba4-8246-293cec7a53bc admin admin] PATH_INFO: `/v3/projects/8b70340671074b2aa4c80384cb6f1c85` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:04.044807 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36c05f2-e640-4ba4-8246-293cec7a53bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b70340671074b2aa4c80384cb6f1c85)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:04.046836 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36c05f2-e640-4ba4-8246-293cec7a53bc admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:04.098496 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b36c05f2-e640-4ba4-8246-293cec7a53bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b70340671074b2aa4c80384cb6f1c85', 'request_id': 'req-b36c05f2-e640-4ba4-8246-293cec7a53bc'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:04.105899 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b36c05f2-e640-4ba4-8246-293cec7a53bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b70340671074b2aa4c80384cb6f1c85', 'request_id': 'req-b36c05f2-e640-4ba4-8246-293cec7a53bc'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:04.118374 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b36c05f2-e640-4ba4-8246-293cec7a53bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b70340671074b2aa4c80384cb6f1c85', 'request_id': 'req-b36c05f2-e640-4ba4-8246-293cec7a53bc'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:04.124681 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:24:04.132320 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b36c05f2-e640-4ba4-8246-293cec7a53bc admin admin] The token cache is being invalidate because project 8b70340671074b2aa4c80384cb6f1c85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:24:04.132864 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b36c05f2-e640-4ba4-8246-293cec7a53bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:04.134232 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b36c05f2-e640-4ba4-8246-293cec7a53bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:04.136097 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1377/2747] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:24:03 2022] DELETE /identity/v3/projects/8b70340671074b2aa4c80384cb6f1c85 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:24:04.142328 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf91f9-9297-4dc2-9d34-11ea62e8d755 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:04.142811 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf91f9-9297-4dc2-9d34-11ea62e8d755 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:04.143117 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf91f9-9297-4dc2-9d34-11ea62e8d755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:04.165312 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-bedf91f9-9297-4dc2-9d34-11ea62e8d755 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:24:04.166671 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-bedf91f9-9297-4dc2-9d34-11ea62e8d755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:04.241474 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1371/2748] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:24:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:24:04.248962 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72911ee3-9486-4873-9f8c-85d69b56d508 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:04.249366 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72911ee3-9486-4873-9f8c-85d69b56d508 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:04.249799 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72911ee3-9486-4873-9f8c-85d69b56d508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:04.271818 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-72911ee3-9486-4873-9f8c-85d69b56d508 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:24:04.273181 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-72911ee3-9486-4873-9f8c-85d69b56d508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:04.286323 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1378/2749] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:24:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:24:04.287476 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf91f9-9297-4dc2-9d34-11ea62e8d755 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:04.292501 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:24:04.304385 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf91f9-9297-4dc2-9d34-11ea62e8d755 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:04.310665 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4e87f-d3f0-40a6-81c0-188762219a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:04.311348 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4e87f-d3f0-40a6-81c0-188762219a51 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:04.311648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4e87f-d3f0-40a6-81c0-188762219a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:04.311920 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4e87f-d3f0-40a6-81c0-188762219a51 admin admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:04.312743 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb4e87f-d3f0-40a6-81c0-188762219a51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:24:04.312958 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb4e87f-d3f0-40a6-81c0-188762219a51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:04.314606 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb4e87f-d3f0-40a6-81c0-188762219a51 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:04.334362 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1372/2750] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:24:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:24:04.335438 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72911ee3-9486-4873-9f8c-85d69b56d508 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:04.352657 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72911ee3-9486-4873-9f8c-85d69b56d508 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:04.358220 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d59ee-2100-4e27-a3e8-73b8b658a3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:04.358902 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d59ee-2100-4e27-a3e8-73b8b658a3ba admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:04.359188 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d59ee-2100-4e27-a3e8-73b8b658a3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:04.359463 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d59ee-2100-4e27-a3e8-73b8b658a3ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:04.360158 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31d59ee-2100-4e27-a3e8-73b8b658a3ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:04.362260 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31d59ee-2100-4e27-a3e8-73b8b658a3ba admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:04.395378 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1379/2751] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:24:04 2022] POST /identity/v3/projects => generated 369 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:24:04.399831 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4e87f-d3f0-40a6-81c0-188762219a51 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:04.411587 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4e87f-d3f0-40a6-81c0-188762219a51 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:04.479463 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e40a0f-f377-4e1d-8aa0-3502885211b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:04.480193 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e40a0f-f377-4e1d-8aa0-3502885211b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:04.480476 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e40a0f-f377-4e1d-8aa0-3502885211b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:04.480747 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e40a0f-f377-4e1d-8aa0-3502885211b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:04.481461 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e40a0f-f377-4e1d-8aa0-3502885211b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:04.483426 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e40a0f-f377-4e1d-8aa0-3502885211b9 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:04.509120 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1373/2752] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:24:04 2022] POST /identity/v3/users => generated 285 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:24:04.510990 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d59ee-2100-4e27-a3e8-73b8b658a3ba admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:04.522681 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d59ee-2100-4e27-a3e8-73b8b658a3ba admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:04.529326 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5390d200-512c-4c0e-b9b7-ccfeefea0929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:04.530301 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5390d200-512c-4c0e-b9b7-ccfeefea0929 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:04.530718 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5390d200-512c-4c0e-b9b7-ccfeefea0929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:04.531122 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5390d200-512c-4c0e-b9b7-ccfeefea0929 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:04.532254 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5390d200-512c-4c0e-b9b7-ccfeefea0929 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:04.534849 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5390d200-512c-4c0e-b9b7-ccfeefea0929 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:04.551582 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1380/2753] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:24:04 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:24:04.557564 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e40a0f-f377-4e1d-8aa0-3502885211b9 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:04.571995 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e40a0f-f377-4e1d-8aa0-3502885211b9 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:04.578210 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc9fae-1f9b-4b00-a916-4cf436f98df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:04.578897 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc9fae-1f9b-4b00-a916-4cf436f98df9 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:04.579181 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc9fae-1f9b-4b00-a916-4cf436f98df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:04.579454 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc9fae-1f9b-4b00-a916-4cf436f98df9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:04.580195 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bc9fae-1f9b-4b00-a916-4cf436f98df9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:04.582198 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bc9fae-1f9b-4b00-a916-4cf436f98df9 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:04.593813 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1374/2754] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:24:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:24:04.598720 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5390d200-512c-4c0e-b9b7-ccfeefea0929 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:04.611076 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5390d200-512c-4c0e-b9b7-ccfeefea0929 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:04.616914 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40735217-9653-46bf-9cd9-0e9181e2f102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:04.618888 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40735217-9653-46bf-9cd9-0e9181e2f102 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:04.619170 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40735217-9653-46bf-9cd9-0e9181e2f102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:04.619430 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40735217-9653-46bf-9cd9-0e9181e2f102 admin admin] PATH_INFO: `/v3/projects/98d7f881222c45ada5de9a6170e69954/users/2619c847e3f24286a60108a316f93a22/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:04.638327 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40735217-9653-46bf-9cd9-0e9181e2f102 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98d7f881222c45ada5de9a6170e69954, user_id=2619c847e3f24286a60108a316f93a22, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:04.641050 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40735217-9653-46bf-9cd9-0e9181e2f102 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:04.655227 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1381/2755] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:24:04 2022] PUT /identity/v3/projects/98d7f881222c45ada5de9a6170e69954/users/2619c847e3f24286a60108a316f93a22/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:24:04.667662 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f8bc67-b4a5-457e-95b3-6626df913f32 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:04.667950 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f8bc67-b4a5-457e-95b3-6626df913f32 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:04.668234 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f8bc67-b4a5-457e-95b3-6626df913f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:04.697885 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-e1f8bc67-b4a5-457e-95b3-6626df913f32 None None] MFA Rules not processed for user `2619c847e3f24286a60108a316f93a22`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:24:04.699130 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-e1f8bc67-b4a5-457e-95b3-6626df913f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:04.769323 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1375/2756] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:24:04 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:24:04.783047 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40735217-9653-46bf-9cd9-0e9181e2f102 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:04.784839 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-40735217-9653-46bf-9cd9-0e9181e2f102 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:04.850232 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40735217-9653-46bf-9cd9-0e9181e2f102 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:04.854713 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12296f4a-d905-4fb3-a26a-b2bceaf3ac91 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26cf962d151b4da7a82c33ef23b342e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:04.855404 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12296f4a-d905-4fb3-a26a-b2bceaf3ac91 service glance] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:04.855683 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12296f4a-d905-4fb3-a26a-b2bceaf3ac91 service glance] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:04.855961 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12296f4a-d905-4fb3-a26a-b2bceaf3ac91 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:04.869582 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12296f4a-d905-4fb3-a26a-b2bceaf3ac91 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:04.872390 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12296f4a-d905-4fb3-a26a-b2bceaf3ac91 service glance] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:04.883052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1382/2757] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:24:04 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:24:06.581746 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f8bc67-b4a5-457e-95b3-6626df913f32 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:06.583558 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-e1f8bc67-b4a5-457e-95b3-6626df913f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:06.649525 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:24:06.653961 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f8bc67-b4a5-457e-95b3-6626df913f32 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:06.659737 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adcea7f-9acd-4705-85da-b7e18ba714b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:06.661295 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adcea7f-9acd-4705-85da-b7e18ba714b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:06.661707 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adcea7f-9acd-4705-85da-b7e18ba714b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:06.662322 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adcea7f-9acd-4705-85da-b7e18ba714b1 admin admin] PATH_INFO: `/v3/users/55bb04a375cd41d7abe02a5f8393b6c2` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:06.664395 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adcea7f-9acd-4705-85da-b7e18ba714b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55bb04a375cd41d7abe02a5f8393b6c2)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:06.668014 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adcea7f-9acd-4705-85da-b7e18ba714b1 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:06.727106 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-9adcea7f-9acd-4705-85da-b7e18ba714b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55bb04a375cd41d7abe02a5f8393b6c2'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:06.734052 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-9adcea7f-9acd-4705-85da-b7e18ba714b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55bb04a375cd41d7abe02a5f8393b6c2'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:06.734892 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-9adcea7f-9acd-4705-85da-b7e18ba714b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55bb04a375cd41d7abe02a5f8393b6c2'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:06.735559 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-9adcea7f-9acd-4705-85da-b7e18ba714b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55bb04a375cd41d7abe02a5f8393b6c2'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:06.744707 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-9adcea7f-9acd-4705-85da-b7e18ba714b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55bb04a375cd41d7abe02a5f8393b6c2'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:06.766899 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1376/2758] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:24:06 2022] DELETE /identity/v3/users/55bb04a375cd41d7abe02a5f8393b6c2 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:24:06.768957 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ee48f-2974-472a-90d5-db50bdb8b741 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:06.769424 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ee48f-2974-472a-90d5-db50bdb8b741 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:06.769661 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ee48f-2974-472a-90d5-db50bdb8b741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:06.792371 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-e92ee48f-2974-472a-90d5-db50bdb8b741 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:24:06.793772 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-e92ee48f-2974-472a-90d5-db50bdb8b741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:06.869832 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1383/2759] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:24:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:24:06.880035 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adcea7f-9acd-4705-85da-b7e18ba714b1 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:06.881708 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-9adcea7f-9acd-4705-85da-b7e18ba714b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:06.946979 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adcea7f-9acd-4705-85da-b7e18ba714b1 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:06.951541 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3a1481-b776-4d86-aea6-2da7829835fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:06.952202 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a1481-b776-4d86-aea6-2da7829835fe service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:06.952486 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a1481-b776-4d86-aea6-2da7829835fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:06.952768 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a1481-b776-4d86-aea6-2da7829835fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:06.955665 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ee48f-2974-472a-90d5-db50bdb8b741 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:06.958215 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-e92ee48f-2974-472a-90d5-db50bdb8b741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:06.964751 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3a1481-b776-4d86-aea6-2da7829835fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:06.966346 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3a1481-b776-4d86-aea6-2da7829835fe service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:06.973329 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ee48f-2974-472a-90d5-db50bdb8b741 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:06.977903 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1377/2760] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:24:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:24:06.979440 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ecb9a1-1faa-4598-ae31-88da496f21ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:06.980408 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ecb9a1-1faa-4598-ae31-88da496f21ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:06.980783 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ecb9a1-1faa-4598-ae31-88da496f21ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:06.981159 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ecb9a1-1faa-4598-ae31-88da496f21ed admin admin] PATH_INFO: `/v3/users/ac486c06b38742198fc56a855d70aa18` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:06.982664 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ecb9a1-1faa-4598-ae31-88da496f21ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac486c06b38742198fc56a855d70aa18)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:06.984708 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ecb9a1-1faa-4598-ae31-88da496f21ed admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:07.029913 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:24:07.048480 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-71ecb9a1-1faa-4598-ae31-88da496f21ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac486c06b38742198fc56a855d70aa18'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:07.057690 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-71ecb9a1-1faa-4598-ae31-88da496f21ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac486c06b38742198fc56a855d70aa18'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:07.058524 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-71ecb9a1-1faa-4598-ae31-88da496f21ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac486c06b38742198fc56a855d70aa18'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:07.059148 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-71ecb9a1-1faa-4598-ae31-88da496f21ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac486c06b38742198fc56a855d70aa18'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:07.066978 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-71ecb9a1-1faa-4598-ae31-88da496f21ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac486c06b38742198fc56a855d70aa18'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:07.086648 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1384/2761] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:24:06 2022] DELETE /identity/v3/users/ac486c06b38742198fc56a855d70aa18 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:24:07.091648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b2940-46ec-4eaf-9de3-567327165bf2 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:07.091940 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b2940-46ec-4eaf-9de3-567327165bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:07.092221 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b2940-46ec-4eaf-9de3-567327165bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:07.110582 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-7f8b2940-46ec-4eaf-9de3-567327165bf2 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:24:07.112174 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-7f8b2940-46ec-4eaf-9de3-567327165bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:07.183336 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1378/2762] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:24:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:24:07.198098 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ecb9a1-1faa-4598-ae31-88da496f21ed admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:07.199725 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-71ecb9a1-1faa-4598-ae31-88da496f21ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:07.264196 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ecb9a1-1faa-4598-ae31-88da496f21ed admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:07.269502 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9045d68-0e08-440d-ac8d-25d2e7c61961 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:07.270029 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9045d68-0e08-440d-ac8d-25d2e7c61961 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:07.270315 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9045d68-0e08-440d-ac8d-25d2e7c61961 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:07.270594 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9045d68-0e08-440d-ac8d-25d2e7c61961 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:07.278129 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b2940-46ec-4eaf-9de3-567327165bf2 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:07.279499 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-7f8b2940-46ec-4eaf-9de3-567327165bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:07.285398 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9045d68-0e08-440d-ac8d-25d2e7c61961 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:07.287644 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9045d68-0e08-440d-ac8d-25d2e7c61961 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:07.297208 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b2940-46ec-4eaf-9de3-567327165bf2 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:07.302048 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1385/2763] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:24:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:24:07.302509 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b28f7-88d9-47b1-8d82-e1ed3c6708ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:07.304364 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b28f7-88d9-47b1-8d82-e1ed3c6708ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:07.304646 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b28f7-88d9-47b1-8d82-e1ed3c6708ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:07.304939 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b28f7-88d9-47b1-8d82-e1ed3c6708ed admin admin] PATH_INFO: `/v3/projects/a50c6488882d4d38994f00a5be02b9c7` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:07.308994 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95b28f7-88d9-47b1-8d82-e1ed3c6708ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=a50c6488882d4d38994f00a5be02b9c7)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:07.310992 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95b28f7-88d9-47b1-8d82-e1ed3c6708ed admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:07.328835 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:24:07.370170 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-a95b28f7-88d9-47b1-8d82-e1ed3c6708ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a50c6488882d4d38994f00a5be02b9c7', 'request_id': 'req-a95b28f7-88d9-47b1-8d82-e1ed3c6708ed'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:07.377688 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-a95b28f7-88d9-47b1-8d82-e1ed3c6708ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a50c6488882d4d38994f00a5be02b9c7', 'request_id': 'req-a95b28f7-88d9-47b1-8d82-e1ed3c6708ed'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:07.387052 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-a95b28f7-88d9-47b1-8d82-e1ed3c6708ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a50c6488882d4d38994f00a5be02b9c7', 'request_id': 'req-a95b28f7-88d9-47b1-8d82-e1ed3c6708ed'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:07.400147 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-a95b28f7-88d9-47b1-8d82-e1ed3c6708ed admin admin] The token cache is being invalidate because project a50c6488882d4d38994f00a5be02b9c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:24:07.400445 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-a95b28f7-88d9-47b1-8d82-e1ed3c6708ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:07.401141 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-a95b28f7-88d9-47b1-8d82-e1ed3c6708ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:07.402927 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1379/2764] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:24:07 2022] DELETE /identity/v3/projects/a50c6488882d4d38994f00a5be02b9c7 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:24:07.408199 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbafe747-736e-405b-a6b5-8a1463dfef0a None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:07.408490 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbafe747-736e-405b-a6b5-8a1463dfef0a None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:07.408763 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbafe747-736e-405b-a6b5-8a1463dfef0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:07.430346 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-dbafe747-736e-405b-a6b5-8a1463dfef0a None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:24:07.431494 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-dbafe747-736e-405b-a6b5-8a1463dfef0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:07.515526 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1386/2765] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:24:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:24:07.524006 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ae09b-8c84-41ff-afb8-2d0d95f735f1 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:07.524315 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ae09b-8c84-41ff-afb8-2d0d95f735f1 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:07.524572 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ae09b-8c84-41ff-afb8-2d0d95f735f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:07.543929 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-259ae09b-8c84-41ff-afb8-2d0d95f735f1 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:24:07.545192 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-259ae09b-8c84-41ff-afb8-2d0d95f735f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:07.553340 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1380/2766] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:24:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:24:07.557692 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbafe747-736e-405b-a6b5-8a1463dfef0a None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:07.569509 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbafe747-736e-405b-a6b5-8a1463dfef0a None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:07.575141 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ad7409-ad2d-4627-94a8-39fdab69ea64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:07.575831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad7409-ad2d-4627-94a8-39fdab69ea64 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:07.576113 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad7409-ad2d-4627-94a8-39fdab69ea64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:07.576386 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad7409-ad2d-4627-94a8-39fdab69ea64 admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:07.577185 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ad7409-ad2d-4627-94a8-39fdab69ea64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:24:07.577491 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ad7409-ad2d-4627-94a8-39fdab69ea64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:07.579203 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ad7409-ad2d-4627-94a8-39fdab69ea64 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:07.592704 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1387/2767] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:24:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:24:07.596853 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ae09b-8c84-41ff-afb8-2d0d95f735f1 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:07.606463 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ae09b-8c84-41ff-afb8-2d0d95f735f1 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:07.611951 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e7c7e-5a81-4fdf-a20b-d10e3cc84142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:07.612605 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e7c7e-5a81-4fdf-a20b-d10e3cc84142 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:07.612885 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e7c7e-5a81-4fdf-a20b-d10e3cc84142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:07.613330 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e7c7e-5a81-4fdf-a20b-d10e3cc84142 admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:07.614115 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494e7c7e-5a81-4fdf-a20b-d10e3cc84142 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:07.616403 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494e7c7e-5a81-4fdf-a20b-d10e3cc84142 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:07.626034 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ad7409-ad2d-4627-94a8-39fdab69ea64 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:07.628368 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-71ad7409-ad2d-4627-94a8-39fdab69ea64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:07.649341 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ad7409-ad2d-4627-94a8-39fdab69ea64 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:07.654855 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1381/2768] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:24:07 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:24:07.659650 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c822711-8a99-4e49-8f54-a3999a05d5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:07.660570 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e7c7e-5a81-4fdf-a20b-d10e3cc84142 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:07.660902 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c822711-8a99-4e49-8f54-a3999a05d5bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:07.661213 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c822711-8a99-4e49-8f54-a3999a05d5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:07.661738 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c822711-8a99-4e49-8f54-a3999a05d5bc admin admin] PATH_INFO: `/v3/projects/15a8108ef16e4046a829ac762b04510b` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:07.663929 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c822711-8a99-4e49-8f54-a3999a05d5bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=15a8108ef16e4046a829ac762b04510b)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:07.666099 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c822711-8a99-4e49-8f54-a3999a05d5bc admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:07.714135 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:24:07.726522 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-7c822711-8a99-4e49-8f54-a3999a05d5bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15a8108ef16e4046a829ac762b04510b', 'request_id': 'req-7c822711-8a99-4e49-8f54-a3999a05d5bc'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:07.733780 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-7c822711-8a99-4e49-8f54-a3999a05d5bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15a8108ef16e4046a829ac762b04510b', 'request_id': 'req-7c822711-8a99-4e49-8f54-a3999a05d5bc'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:07.738433 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e7c7e-5a81-4fdf-a20b-d10e3cc84142 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:07.742371 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-7c822711-8a99-4e49-8f54-a3999a05d5bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15a8108ef16e4046a829ac762b04510b', 'request_id': 'req-7c822711-8a99-4e49-8f54-a3999a05d5bc'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:07.753630 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-7c822711-8a99-4e49-8f54-a3999a05d5bc admin admin] The token cache is being invalidate because project 15a8108ef16e4046a829ac762b04510b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:24:07.753918 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-7c822711-8a99-4e49-8f54-a3999a05d5bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:07.754567 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-7c822711-8a99-4e49-8f54-a3999a05d5bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:07.756459 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1388/2769] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:24:07 2022] DELETE /identity/v3/projects/15a8108ef16e4046a829ac762b04510b => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:24:07.760409 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c822711-8a99-4e49-8f54-a3999a05d5bc admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:07.762019 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-7c822711-8a99-4e49-8f54-a3999a05d5bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:07.807389 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4af89-1868-47e1-9c1e-b8588d215caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:07.808085 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4af89-1868-47e1-9c1e-b8588d215caf admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:07.808369 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4af89-1868-47e1-9c1e-b8588d215caf admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:07.808648 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4af89-1868-47e1-9c1e-b8588d215caf admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:07.809381 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e4af89-1868-47e1-9c1e-b8588d215caf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:07.811352 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e4af89-1868-47e1-9c1e-b8588d215caf admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:07.833542 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1382/2770] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:24:07 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:24:07.836973 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c822711-8a99-4e49-8f54-a3999a05d5bc admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:07.838016 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4af89-1868-47e1-9c1e-b8588d215caf admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:07.839620 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-f7e4af89-1868-47e1-9c1e-b8588d215caf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:07.840382 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac6d02-dc1f-4201-874c-4c0a7cc0f8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:07.841294 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac6d02-dc1f-4201-874c-4c0a7cc0f8ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:07.841575 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac6d02-dc1f-4201-874c-4c0a7cc0f8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:07.841851 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac6d02-dc1f-4201-874c-4c0a7cc0f8ca admin admin] PATH_INFO: `/v3/users/c022de91e2984dcb8370d2cfe506a799` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:07.843049 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac6d02-dc1f-4201-874c-4c0a7cc0f8ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=c022de91e2984dcb8370d2cfe506a799)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:07.845016 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac6d02-dc1f-4201-874c-4c0a7cc0f8ca admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:07.851749 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4af89-1868-47e1-9c1e-b8588d215caf admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:07.856732 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f8d734-bab5-455d-b10e-1fe0106dd696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:07.857478 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f8d734-bab5-455d-b10e-1fe0106dd696 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:07.858586 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f8d734-bab5-455d-b10e-1fe0106dd696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:07.858586 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f8d734-bab5-455d-b10e-1fe0106dd696 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:07.859020 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f8d734-bab5-455d-b10e-1fe0106dd696 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:07.861255 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f8d734-bab5-455d-b10e-1fe0106dd696 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:07.868143 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:24:07.873788 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1383/2771] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:24:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:24:07.878278 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f8d734-bab5-455d-b10e-1fe0106dd696 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:07.890974 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f8d734-bab5-455d-b10e-1fe0106dd696 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:07.895960 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8554c4-a504-44e6-8684-abc28cde9700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:07.896791 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8554c4-a504-44e6-8684-abc28cde9700 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:07.897119 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8554c4-a504-44e6-8684-abc28cde9700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:07.897397 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8554c4-a504-44e6-8684-abc28cde9700 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:07.898189 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8554c4-a504-44e6-8684-abc28cde9700 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:07.899799 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8554c4-a504-44e6-8684-abc28cde9700 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:07.900113 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-76ac6d02-dc1f-4201-874c-4c0a7cc0f8ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c022de91e2984dcb8370d2cfe506a799'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:07.907390 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-76ac6d02-dc1f-4201-874c-4c0a7cc0f8ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c022de91e2984dcb8370d2cfe506a799'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:07.908090 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-76ac6d02-dc1f-4201-874c-4c0a7cc0f8ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c022de91e2984dcb8370d2cfe506a799'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:07.908710 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-76ac6d02-dc1f-4201-874c-4c0a7cc0f8ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c022de91e2984dcb8370d2cfe506a799'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:07.911082 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1384/2772] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:24:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:24:07.916273 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8554c4-a504-44e6-8684-abc28cde9700 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:07.918345 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-bc8554c4-a504-44e6-8684-abc28cde9700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:07.922001 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-76ac6d02-dc1f-4201-874c-4c0a7cc0f8ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c022de91e2984dcb8370d2cfe506a799'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:07.939348 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8554c4-a504-44e6-8684-abc28cde9700 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:07.944897 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578172e9-a0d3-4f23-9b09-34df11023f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:07.945927 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578172e9-a0d3-4f23-9b09-34df11023f24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:07.946061 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1389/2773] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:24:07 2022] DELETE /identity/v3/users/c022de91e2984dcb8370d2cfe506a799 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:24:07.946221 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578172e9-a0d3-4f23-9b09-34df11023f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:07.946505 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578172e9-a0d3-4f23-9b09-34df11023f24 admin admin] PATH_INFO: `/v3/projects/c9f86d6febe04e8f9745d787914b6d5c/users/4d867759b4f249a393598964774a22fa/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:07.967573 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578172e9-a0d3-4f23-9b09-34df11023f24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9f86d6febe04e8f9745d787914b6d5c, user_id=4d867759b4f249a393598964774a22fa, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:07.970355 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578172e9-a0d3-4f23-9b09-34df11023f24 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:07.987526 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1385/2774] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:24:07 2022] PUT /identity/v3/projects/c9f86d6febe04e8f9745d787914b6d5c/users/4d867759b4f249a393598964774a22fa/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:24:07.994178 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de896364-06d6-45f3-8804-314794e04b22 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:07.994808 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de896364-06d6-45f3-8804-314794e04b22 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:07.995142 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de896364-06d6-45f3-8804-314794e04b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:08.019392 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-de896364-06d6-45f3-8804-314794e04b22 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:24:08.021029 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-de896364-06d6-45f3-8804-314794e04b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:08.072448 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:24:08.101567 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1390/2775] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:24:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:24:08.117165 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578172e9-a0d3-4f23-9b09-34df11023f24 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:08.122341 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-578172e9-a0d3-4f23-9b09-34df11023f24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:08.182901 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578172e9-a0d3-4f23-9b09-34df11023f24 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:08.187056 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08adac8d-f5ed-41ee-9cea-1574bb8bef85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:08.187879 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08adac8d-f5ed-41ee-9cea-1574bb8bef85 service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:08.188229 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08adac8d-f5ed-41ee-9cea-1574bb8bef85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:08.188598 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08adac8d-f5ed-41ee-9cea-1574bb8bef85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:08.200187 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08adac8d-f5ed-41ee-9cea-1574bb8bef85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:08.202492 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08adac8d-f5ed-41ee-9cea-1574bb8bef85 service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:08.216077 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1386/2776] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:24:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:24:08.288507 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de896364-06d6-45f3-8804-314794e04b22 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:08.290068 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-de896364-06d6-45f3-8804-314794e04b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:08.304649 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de896364-06d6-45f3-8804-314794e04b22 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:08.311985 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123078c7-8277-4113-a99f-d026696f372a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:08.312900 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123078c7-8277-4113-a99f-d026696f372a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:08.313193 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123078c7-8277-4113-a99f-d026696f372a admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:08.313462 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123078c7-8277-4113-a99f-d026696f372a admin admin] PATH_INFO: `/v3/projects/aa905a32ff4b49679051eec9159fd91e` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:08.314719 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123078c7-8277-4113-a99f-d026696f372a admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa905a32ff4b49679051eec9159fd91e)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:08.317099 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123078c7-8277-4113-a99f-d026696f372a admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:08.385596 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-123078c7-8277-4113-a99f-d026696f372a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa905a32ff4b49679051eec9159fd91e', 'request_id': 'req-123078c7-8277-4113-a99f-d026696f372a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:08.393847 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-123078c7-8277-4113-a99f-d026696f372a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa905a32ff4b49679051eec9159fd91e', 'request_id': 'req-123078c7-8277-4113-a99f-d026696f372a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:08.410740 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-123078c7-8277-4113-a99f-d026696f372a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa905a32ff4b49679051eec9159fd91e', 'request_id': 'req-123078c7-8277-4113-a99f-d026696f372a'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:08.422281 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-123078c7-8277-4113-a99f-d026696f372a admin admin] The token cache is being invalidate because project aa905a32ff4b49679051eec9159fd91e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:24:08.422641 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-123078c7-8277-4113-a99f-d026696f372a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:08.423364 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-123078c7-8277-4113-a99f-d026696f372a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:08.425016 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1391/2777] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:24:08 2022] DELETE /identity/v3/projects/aa905a32ff4b49679051eec9159fd91e => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:24:08.430030 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba13e9-75d5-47f1-9bf9-07ebb548ad63 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:08.430324 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba13e9-75d5-47f1-9bf9-07ebb548ad63 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:08.430603 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba13e9-75d5-47f1-9bf9-07ebb548ad63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:08.452137 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-bdba13e9-75d5-47f1-9bf9-07ebb548ad63 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:24:08.453511 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-bdba13e9-75d5-47f1-9bf9-07ebb548ad63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:08.560289 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1387/2778] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:24:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:24:08.562865 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d0809-aba2-47f3-9e8a-4fdbdffc5523 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:08.563190 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d0809-aba2-47f3-9e8a-4fdbdffc5523 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:08.563445 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d0809-aba2-47f3-9e8a-4fdbdffc5523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:08.587994 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-368d0809-aba2-47f3-9e8a-4fdbdffc5523 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:24:08.587994 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-368d0809-aba2-47f3-9e8a-4fdbdffc5523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:08.596602 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1392/2779] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:24:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:24:08.601644 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba13e9-75d5-47f1-9bf9-07ebb548ad63 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:08.614631 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdba13e9-75d5-47f1-9bf9-07ebb548ad63 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:08.625111 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b8122-df20-43ff-9e97-8b31f2bc221f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:08.625976 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b8122-df20-43ff-9e97-8b31f2bc221f admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:08.626271 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b8122-df20-43ff-9e97-8b31f2bc221f admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:08.626593 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b8122-df20-43ff-9e97-8b31f2bc221f admin admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:08.627387 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79b8122-df20-43ff-9e97-8b31f2bc221f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:24:08.627749 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79b8122-df20-43ff-9e97-8b31f2bc221f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:08.629464 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79b8122-df20-43ff-9e97-8b31f2bc221f admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:08.649082 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1388/2780] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:24:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:24:08.654203 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d0809-aba2-47f3-9e8a-4fdbdffc5523 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:08.665598 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d0809-aba2-47f3-9e8a-4fdbdffc5523 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:08.671513 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a15a3db-8294-4ce8-a406-1737b8204ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:08.672327 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15a3db-8294-4ce8-a406-1737b8204ad9 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:08.672687 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15a3db-8294-4ce8-a406-1737b8204ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:08.673120 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15a3db-8294-4ce8-a406-1737b8204ad9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:08.674046 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a15a3db-8294-4ce8-a406-1737b8204ad9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:08.676335 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a15a3db-8294-4ce8-a406-1737b8204ad9 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:08.712205 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1393/2781] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:24:08 2022] POST /identity/v3/projects => generated 353 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:24:08.716747 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b8122-df20-43ff-9e97-8b31f2bc221f admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:08.727980 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b8122-df20-43ff-9e97-8b31f2bc221f admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:08.785796 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:24:08.794172 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae69959e-8a7c-42b1-ab6f-af1d65d21331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:08.794863 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae69959e-8a7c-42b1-ab6f-af1d65d21331 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:08.795148 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae69959e-8a7c-42b1-ab6f-af1d65d21331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:08.795424 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae69959e-8a7c-42b1-ab6f-af1d65d21331 admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:08.796141 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae69959e-8a7c-42b1-ab6f-af1d65d21331 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:08.798161 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae69959e-8a7c-42b1-ab6f-af1d65d21331 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:08.819961 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1389/2782] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:24:08 2022] POST /identity/v3/users => generated 277 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:24:08.824941 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a15a3db-8294-4ce8-a406-1737b8204ad9 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:08.838550 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a15a3db-8294-4ce8-a406-1737b8204ad9 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:08.843577 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e235b068-d12e-41b7-8d89-615d2f9cce7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:08.844334 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e235b068-d12e-41b7-8d89-615d2f9cce7a admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:08.844684 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e235b068-d12e-41b7-8d89-615d2f9cce7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:08.845056 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e235b068-d12e-41b7-8d89-615d2f9cce7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:08.845855 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e235b068-d12e-41b7-8d89-615d2f9cce7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:08.847590 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e235b068-d12e-41b7-8d89-615d2f9cce7a admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:08.866044 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1394/2783] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:24:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:24:08.868069 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae69959e-8a7c-42b1-ab6f-af1d65d21331 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:08.881190 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae69959e-8a7c-42b1-ab6f-af1d65d21331 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:08.886919 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f7c40-428e-48d7-a406-10915cff7ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:08.887726 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f7c40-428e-48d7-a406-10915cff7ccb admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:08.888950 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f7c40-428e-48d7-a406-10915cff7ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:08.889352 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f7c40-428e-48d7-a406-10915cff7ccb admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:08.890794 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0f7c40-428e-48d7-a406-10915cff7ccb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:08.892719 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0f7c40-428e-48d7-a406-10915cff7ccb admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:08.907585 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1390/2784] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:24:08 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:24:08.914161 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e235b068-d12e-41b7-8d89-615d2f9cce7a admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:08.932824 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e235b068-d12e-41b7-8d89-615d2f9cce7a admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:08.940513 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad89114-fa6b-40d8-bae5-12eabc12fa03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:08.942156 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad89114-fa6b-40d8-bae5-12eabc12fa03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:08.942707 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad89114-fa6b-40d8-bae5-12eabc12fa03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:08.943096 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad89114-fa6b-40d8-bae5-12eabc12fa03 admin admin] PATH_INFO: `/v3/projects/6efc0f57c373472d89762f0410e76f4a/users/22e1f20affeb4c6a9e344a715d2846b0/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:08.968285 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad89114-fa6b-40d8-bae5-12eabc12fa03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6efc0f57c373472d89762f0410e76f4a, user_id=22e1f20affeb4c6a9e344a715d2846b0, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:08.973762 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad89114-fa6b-40d8-bae5-12eabc12fa03 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:08.997766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1395/2785] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:24:08 2022] PUT /identity/v3/projects/6efc0f57c373472d89762f0410e76f4a/users/22e1f20affeb4c6a9e344a715d2846b0/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:24:09.008906 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770885a-a765-45e8-ab15-c98fbd0f3271 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:09.009363 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770885a-a765-45e8-ab15-c98fbd0f3271 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:09.010005 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770885a-a765-45e8-ab15-c98fbd0f3271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:09.041286 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-7770885a-a765-45e8-ab15-c98fbd0f3271 None None] MFA Rules not processed for user `22e1f20affeb4c6a9e344a715d2846b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:24:09.042556 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-7770885a-a765-45e8-ab15-c98fbd0f3271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:09.144523 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1391/2786] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:24:09 2022] POST /identity/v3/auth/tokens => generated 3220 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:24:09.181530 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad89114-fa6b-40d8-bae5-12eabc12fa03 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:09.183275 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-bad89114-fa6b-40d8-bae5-12eabc12fa03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:09.272383 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad89114-fa6b-40d8-bae5-12eabc12fa03 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:09.276725 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6ef72e-5c4e-4275-a703-add4c01e5b7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:09.277642 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ef72e-5c4e-4275-a703-add4c01e5b7c service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:09.277642 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ef72e-5c4e-4275-a703-add4c01e5b7c service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:09.277856 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ef72e-5c4e-4275-a703-add4c01e5b7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:09.287602 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6ef72e-5c4e-4275-a703-add4c01e5b7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:09.289080 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6ef72e-5c4e-4275-a703-add4c01e5b7c service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:09.301220 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1396/2787] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:24:09 2022] GET /identity/v3/auth/tokens => generated 3220 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:24:09.314021 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770885a-a765-45e8-ab15-c98fbd0f3271 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:09.395810 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770885a-a765-45e8-ab15-c98fbd0f3271 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:09.401747 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86dc8d8-c68f-4651-9bf8-30ce3a8fb495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:09.403189 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86dc8d8-c68f-4651-9bf8-30ce3a8fb495 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:09.403705 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86dc8d8-c68f-4651-9bf8-30ce3a8fb495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:09.404339 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86dc8d8-c68f-4651-9bf8-30ce3a8fb495 admin admin] PATH_INFO: `/v3/users/22e1f20affeb4c6a9e344a715d2846b0` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:09.414540 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86dc8d8-c68f-4651-9bf8-30ce3a8fb495 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22e1f20affeb4c6a9e344a715d2846b0)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:09.416752 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86dc8d8-c68f-4651-9bf8-30ce3a8fb495 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:09.473432 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-b86dc8d8-c68f-4651-9bf8-30ce3a8fb495 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22e1f20affeb4c6a9e344a715d2846b0'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:09.495763 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-b86dc8d8-c68f-4651-9bf8-30ce3a8fb495 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22e1f20affeb4c6a9e344a715d2846b0'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:09.496205 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-b86dc8d8-c68f-4651-9bf8-30ce3a8fb495 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22e1f20affeb4c6a9e344a715d2846b0'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:09.496805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-b86dc8d8-c68f-4651-9bf8-30ce3a8fb495 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22e1f20affeb4c6a9e344a715d2846b0'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:09.509714 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-b86dc8d8-c68f-4651-9bf8-30ce3a8fb495 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22e1f20affeb4c6a9e344a715d2846b0'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:09.513200 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:24:09.541179 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1392/2788] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:24:09 2022] DELETE /identity/v3/users/22e1f20affeb4c6a9e344a715d2846b0 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:24:09.545782 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8d708-12a8-4c15-a873-f7d1a8da5be2 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:09.546187 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8d708-12a8-4c15-a873-f7d1a8da5be2 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:09.546556 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8d708-12a8-4c15-a873-f7d1a8da5be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:09.559523 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:24:09.579691 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-caa8d708-12a8-4c15-a873-f7d1a8da5be2 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:24:09.582247 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-caa8d708-12a8-4c15-a873-f7d1a8da5be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:09.661108 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1397/2789] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:24:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:24:09.671522 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86dc8d8-c68f-4651-9bf8-30ce3a8fb495 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:09.673544 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-b86dc8d8-c68f-4651-9bf8-30ce3a8fb495 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:09.761283 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86dc8d8-c68f-4651-9bf8-30ce3a8fb495 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:09.767220 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced2a579-ba51-41b7-a006-9e8bc0dbb282 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:09.768346 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2a579-ba51-41b7-a006-9e8bc0dbb282 service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:09.768788 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2a579-ba51-41b7-a006-9e8bc0dbb282 service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:09.769615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2a579-ba51-41b7-a006-9e8bc0dbb282 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:09.793061 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced2a579-ba51-41b7-a006-9e8bc0dbb282 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:09.795150 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced2a579-ba51-41b7-a006-9e8bc0dbb282 service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:09.815644 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1393/2790] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:24:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:24:10.206435 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8d708-12a8-4c15-a873-f7d1a8da5be2 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:10.209637 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-caa8d708-12a8-4c15-a873-f7d1a8da5be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:10.224247 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8d708-12a8-4c15-a873-f7d1a8da5be2 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:10.230098 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98777637-7e3f-4689-88a0-7b3a5996d1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:10.231195 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98777637-7e3f-4689-88a0-7b3a5996d1e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:10.231600 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98777637-7e3f-4689-88a0-7b3a5996d1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:10.231947 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98777637-7e3f-4689-88a0-7b3a5996d1e4 admin admin] PATH_INFO: `/v3/projects/6efc0f57c373472d89762f0410e76f4a` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:10.233295 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98777637-7e3f-4689-88a0-7b3a5996d1e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6efc0f57c373472d89762f0410e76f4a)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:10.235169 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98777637-7e3f-4689-88a0-7b3a5996d1e4 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:10.291538 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-98777637-7e3f-4689-88a0-7b3a5996d1e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6efc0f57c373472d89762f0410e76f4a', 'request_id': 'req-98777637-7e3f-4689-88a0-7b3a5996d1e4'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:10.297858 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-98777637-7e3f-4689-88a0-7b3a5996d1e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6efc0f57c373472d89762f0410e76f4a', 'request_id': 'req-98777637-7e3f-4689-88a0-7b3a5996d1e4'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:10.307924 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-98777637-7e3f-4689-88a0-7b3a5996d1e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6efc0f57c373472d89762f0410e76f4a', 'request_id': 'req-98777637-7e3f-4689-88a0-7b3a5996d1e4'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:10.321439 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-98777637-7e3f-4689-88a0-7b3a5996d1e4 admin admin] The token cache is being invalidate because project 6efc0f57c373472d89762f0410e76f4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:24:10.321987 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-98777637-7e3f-4689-88a0-7b3a5996d1e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:10.322852 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-98777637-7e3f-4689-88a0-7b3a5996d1e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:10.328894 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1398/2791] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:24:10 2022] DELETE /identity/v3/projects/6efc0f57c373472d89762f0410e76f4a => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:24:10.331175 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf46cb1-3ce3-45ef-a5c0-3573c93ecf69 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:10.331520 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf46cb1-3ce3-45ef-a5c0-3573c93ecf69 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:10.331827 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf46cb1-3ce3-45ef-a5c0-3573c93ecf69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:10.355002 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-ebf46cb1-3ce3-45ef-a5c0-3573c93ecf69 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:24:10.356153 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-ebf46cb1-3ce3-45ef-a5c0-3573c93ecf69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:10.432646 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1394/2792] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:24:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:24:10.440640 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497eb37-a34f-4bd6-8a40-1c84eba38206 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:10.441259 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497eb37-a34f-4bd6-8a40-1c84eba38206 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:10.441613 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497eb37-a34f-4bd6-8a40-1c84eba38206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:10.462857 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-c497eb37-a34f-4bd6-8a40-1c84eba38206 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:24:10.464165 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-c497eb37-a34f-4bd6-8a40-1c84eba38206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:10.479362 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1399/2793] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:24:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:24:10.481106 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf46cb1-3ce3-45ef-a5c0-3573c93ecf69 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:10.491995 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf46cb1-3ce3-45ef-a5c0-3573c93ecf69 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:10.501638 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819af4a3-8c32-4ae2-b47e-96e7cfa07f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:10.501638 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819af4a3-8c32-4ae2-b47e-96e7cfa07f9a admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:10.501638 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819af4a3-8c32-4ae2-b47e-96e7cfa07f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:10.501638 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819af4a3-8c32-4ae2-b47e-96e7cfa07f9a admin admin] PATH_INFO: `/v3/domains` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:10.502581 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819af4a3-8c32-4ae2-b47e-96e7cfa07f9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:24:10.502913 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819af4a3-8c32-4ae2-b47e-96e7cfa07f9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:10.504626 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819af4a3-8c32-4ae2-b47e-96e7cfa07f9a admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:10.518926 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1395/2794] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:24:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:24:10.523421 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497eb37-a34f-4bd6-8a40-1c84eba38206 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:10.534005 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497eb37-a34f-4bd6-8a40-1c84eba38206 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:10.539476 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a5632d-bba0-494d-b133-5bb73bc5989a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:10.540007 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5632d-bba0-494d-b133-5bb73bc5989a admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:10.540327 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5632d-bba0-494d-b133-5bb73bc5989a admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:10.540604 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5632d-bba0-494d-b133-5bb73bc5989a admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:10.541292 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a5632d-bba0-494d-b133-5bb73bc5989a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:10.543583 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a5632d-bba0-494d-b133-5bb73bc5989a admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:10.578694 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1400/2795] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:24:10 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:24:10.583539 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819af4a3-8c32-4ae2-b47e-96e7cfa07f9a admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:10.594945 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819af4a3-8c32-4ae2-b47e-96e7cfa07f9a admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:10.695750 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac943a52-8318-4958-8d0c-1162a5c52833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:10.696459 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac943a52-8318-4958-8d0c-1162a5c52833 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:10.696748 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac943a52-8318-4958-8d0c-1162a5c52833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:10.697075 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac943a52-8318-4958-8d0c-1162a5c52833 admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:10.697717 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac943a52-8318-4958-8d0c-1162a5c52833 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:10.699728 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac943a52-8318-4958-8d0c-1162a5c52833 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:10.739470 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a5632d-bba0-494d-b133-5bb73bc5989a admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:10.748101 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1396/2796] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:24:10 2022] POST /identity/v3/users => generated 292 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:24:10.752395 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a5632d-bba0-494d-b133-5bb73bc5989a admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:10.761636 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e782173-b8c2-40db-8339-053359dbdc64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:10.762712 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e782173-b8c2-40db-8339-053359dbdc64 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:10.762890 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e782173-b8c2-40db-8339-053359dbdc64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:10.763123 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e782173-b8c2-40db-8339-053359dbdc64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:10.763828 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e782173-b8c2-40db-8339-053359dbdc64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:10.765548 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e782173-b8c2-40db-8339-053359dbdc64 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:10.781879 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1401/2797] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:24:10.783239 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac943a52-8318-4958-8d0c-1162a5c52833 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:10.789273 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:24:10.797407 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac943a52-8318-4958-8d0c-1162a5c52833 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:10.804865 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74481b0-c452-4982-82cb-30b68c33ce34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:10.805687 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74481b0-c452-4982-82cb-30b68c33ce34 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:10.805974 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74481b0-c452-4982-82cb-30b68c33ce34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:10.806253 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74481b0-c452-4982-82cb-30b68c33ce34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:10.807009 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74481b0-c452-4982-82cb-30b68c33ce34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:10.808692 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74481b0-c452-4982-82cb-30b68c33ce34 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:10.827364 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1397/2798] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:24:10.828510 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e782173-b8c2-40db-8339-053359dbdc64 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:10.841325 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e782173-b8c2-40db-8339-053359dbdc64 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:10.847159 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1163a-facf-4f25-8f42-fd044d5bb625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:10.848372 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1163a-facf-4f25-8f42-fd044d5bb625 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:10.848761 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1163a-facf-4f25-8f42-fd044d5bb625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:10.849229 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1163a-facf-4f25-8f42-fd044d5bb625 admin admin] PATH_INFO: `/v3/projects/8e903efc6e8f487d950e079eff67922c/users/eec869d87067467d81611326c4f773ed/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:10.889232 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c1163a-facf-4f25-8f42-fd044d5bb625 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e903efc6e8f487d950e079eff67922c, user_id=eec869d87067467d81611326c4f773ed, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:10.891907 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c1163a-facf-4f25-8f42-fd044d5bb625 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:10.913602 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1402/2799] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:24:10 2022] PUT /identity/v3/projects/8e903efc6e8f487d950e079eff67922c/users/eec869d87067467d81611326c4f773ed/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:24:10.925531 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b631f3-e6f7-46aa-84ea-2b7e9a6f3e9e None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:10.925947 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b631f3-e6f7-46aa-84ea-2b7e9a6f3e9e None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:10.926321 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b631f3-e6f7-46aa-84ea-2b7e9a6f3e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:10.967526 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-39b631f3-e6f7-46aa-84ea-2b7e9a6f3e9e None None] MFA Rules not processed for user `eec869d87067467d81611326c4f773ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:24:10.968916 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-39b631f3-e6f7-46aa-84ea-2b7e9a6f3e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:11.107220 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1398/2800] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:24:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:24:11.115203 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1163a-facf-4f25-8f42-fd044d5bb625 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:11.155617 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:24:11.236767 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1163a-facf-4f25-8f42-fd044d5bb625 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:11.242585 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f99a758-68d5-4a04-af6a-c5f3f87b90a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:11.243214 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f99a758-68d5-4a04-af6a-c5f3f87b90a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:11.243497 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f99a758-68d5-4a04-af6a-c5f3f87b90a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:11.243766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f99a758-68d5-4a04-af6a-c5f3f87b90a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:11.244647 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f99a758-68d5-4a04-af6a-c5f3f87b90a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:11.247308 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f99a758-68d5-4a04-af6a-c5f3f87b90a9 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:11.304363 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1403/2801] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:24:11 2022] POST /identity/v3/projects => generated 383 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:24:11.308164 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b631f3-e6f7-46aa-84ea-2b7e9a6f3e9e None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:11.322478 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b631f3-e6f7-46aa-84ea-2b7e9a6f3e9e None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:11.431701 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cee924-f2d3-427e-b270-04ca3df7ce7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:11.432404 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cee924-f2d3-427e-b270-04ca3df7ce7f admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:11.432686 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cee924-f2d3-427e-b270-04ca3df7ce7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:11.433000 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cee924-f2d3-427e-b270-04ca3df7ce7f admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:11.433730 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cee924-f2d3-427e-b270-04ca3df7ce7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:11.435969 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cee924-f2d3-427e-b270-04ca3df7ce7f admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:11.462347 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1399/2802] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:24:11 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:24:11.467358 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f99a758-68d5-4a04-af6a-c5f3f87b90a9 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:11.482375 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f99a758-68d5-4a04-af6a-c5f3f87b90a9 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:11.489653 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3beb7d-9dbd-4234-b63a-a4bf217624f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:11.490872 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3beb7d-9dbd-4234-b63a-a4bf217624f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:11.490872 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3beb7d-9dbd-4234-b63a-a4bf217624f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:11.493005 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3beb7d-9dbd-4234-b63a-a4bf217624f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:11.494085 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3beb7d-9dbd-4234-b63a-a4bf217624f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:11.495996 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3beb7d-9dbd-4234-b63a-a4bf217624f7 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:11.510153 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1404/2803] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:24:11 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:24:11.514538 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cee924-f2d3-427e-b270-04ca3df7ce7f admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:11.528330 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cee924-f2d3-427e-b270-04ca3df7ce7f admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:11.535050 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5597fa-152f-4856-bf0f-c6738eeaf9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:11.536276 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5597fa-152f-4856-bf0f-c6738eeaf9f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:11.536561 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5597fa-152f-4856-bf0f-c6738eeaf9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:11.538013 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5597fa-152f-4856-bf0f-c6738eeaf9f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:11.539469 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5597fa-152f-4856-bf0f-c6738eeaf9f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:11.540965 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5597fa-152f-4856-bf0f-c6738eeaf9f1 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:11.553242 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1400/2804] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:24:11 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:24:11.557322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3beb7d-9dbd-4234-b63a-a4bf217624f7 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:11.567877 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3beb7d-9dbd-4234-b63a-a4bf217624f7 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:11.573402 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0508b2-40f7-460a-832f-d4d84aeb48ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:11.574538 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0508b2-40f7-460a-832f-d4d84aeb48ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:11.574911 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0508b2-40f7-460a-832f-d4d84aeb48ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:11.575281 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0508b2-40f7-460a-832f-d4d84aeb48ff admin admin] PATH_INFO: `/v3/projects/935a2203fe1a4234963a18b6a048f163/users/3e7b5d6f8b71481384436c953a3775a5/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:11.592868 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0508b2-40f7-460a-832f-d4d84aeb48ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=935a2203fe1a4234963a18b6a048f163, user_id=3e7b5d6f8b71481384436c953a3775a5, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:11.595818 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0508b2-40f7-460a-832f-d4d84aeb48ff admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:11.617880 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1405/2805] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:24:11 2022] PUT /identity/v3/projects/935a2203fe1a4234963a18b6a048f163/users/3e7b5d6f8b71481384436c953a3775a5/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:24:11.630909 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29471f35-a38f-4ccb-bc01-a74966dc7d16 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:11.631303 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29471f35-a38f-4ccb-bc01-a74966dc7d16 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:11.631656 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29471f35-a38f-4ccb-bc01-a74966dc7d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:11.696499 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-29471f35-a38f-4ccb-bc01-a74966dc7d16 None None] MFA Rules not processed for user `3e7b5d6f8b71481384436c953a3775a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:24:11.699455 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-29471f35-a38f-4ccb-bc01-a74966dc7d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:11.814919 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:24:11.841740 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1401/2806] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:24:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:24:11.846760 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0508b2-40f7-460a-832f-d4d84aeb48ff admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:11.955208 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0508b2-40f7-460a-832f-d4d84aeb48ff admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:11.964211 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41dc09d-e29a-44c1-bbc6-4fa70e34aa5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:11.964901 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41dc09d-e29a-44c1-bbc6-4fa70e34aa5d admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:11.965225 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41dc09d-e29a-44c1-bbc6-4fa70e34aa5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:11.965554 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41dc09d-e29a-44c1-bbc6-4fa70e34aa5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:11.966359 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41dc09d-e29a-44c1-bbc6-4fa70e34aa5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:11.973517 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41dc09d-e29a-44c1-bbc6-4fa70e34aa5d admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:12.028256 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1406/2807] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:24:11 2022] POST /identity/v3/projects => generated 383 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:24:12.029728 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29471f35-a38f-4ccb-bc01-a74966dc7d16 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:12.048514 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29471f35-a38f-4ccb-bc01-a74966dc7d16 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:12.128766 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a4f07-5fb1-48af-88d4-402448fd630d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:12.129513 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a4f07-5fb1-48af-88d4-402448fd630d admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:12.129955 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a4f07-5fb1-48af-88d4-402448fd630d admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:12.130097 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a4f07-5fb1-48af-88d4-402448fd630d admin admin] PATH_INFO: `/v3/users` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:12.131066 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50a4f07-5fb1-48af-88d4-402448fd630d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:12.133309 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50a4f07-5fb1-48af-88d4-402448fd630d admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:12.159396 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1402/2808] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:24:12 2022] POST /identity/v3/users => generated 298 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:24:12.163332 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41dc09d-e29a-44c1-bbc6-4fa70e34aa5d admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:12.186126 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41dc09d-e29a-44c1-bbc6-4fa70e34aa5d admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:12.191412 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47f01dd-45ca-466f-aef0-3be436057d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:12.192103 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f01dd-45ca-466f-aef0-3be436057d9b admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:12.192382 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f01dd-45ca-466f-aef0-3be436057d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:12.192990 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f01dd-45ca-466f-aef0-3be436057d9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:12.193818 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47f01dd-45ca-466f-aef0-3be436057d9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:12.195508 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47f01dd-45ca-466f-aef0-3be436057d9b admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:12.211269 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1407/2809] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:24:12.215914 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a4f07-5fb1-48af-88d4-402448fd630d admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:12.232652 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a4f07-5fb1-48af-88d4-402448fd630d admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:12.238007 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca79905-697e-4e74-b471-8a75ce002e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:12.239001 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca79905-697e-4e74-b471-8a75ce002e3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:12.239284 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca79905-697e-4e74-b471-8a75ce002e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:12.239557 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca79905-697e-4e74-b471-8a75ce002e3d admin admin] PATH_INFO: `/v3/projects/9c235ff08ff049e19dafd7c27df823e1/users/ffc92062ef424bec8c3da74d07fd6237/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:12.260683 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca79905-697e-4e74-b471-8a75ce002e3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c235ff08ff049e19dafd7c27df823e1, user_id=ffc92062ef424bec8c3da74d07fd6237, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:12.263921 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca79905-697e-4e74-b471-8a75ce002e3d admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:12.286684 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1403/2810] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:24:12 2022] PUT /identity/v3/projects/9c235ff08ff049e19dafd7c27df823e1/users/ffc92062ef424bec8c3da74d07fd6237/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:24:12.291135 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47f01dd-45ca-466f-aef0-3be436057d9b admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:12.302326 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47f01dd-45ca-466f-aef0-3be436057d9b admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:12.373182 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:24:12.403080 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0978c0dd-2c94-43fe-83cd-0344abb9b6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:12.403813 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0978c0dd-2c94-43fe-83cd-0344abb9b6b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:12.405052 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0978c0dd-2c94-43fe-83cd-0344abb9b6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:12.405345 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0978c0dd-2c94-43fe-83cd-0344abb9b6b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:12.406142 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0978c0dd-2c94-43fe-83cd-0344abb9b6b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:12.407792 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0978c0dd-2c94-43fe-83cd-0344abb9b6b5 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:12.422888 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1408/2811] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:24:12.424289 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca79905-697e-4e74-b471-8a75ce002e3d admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:12.435602 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca79905-697e-4e74-b471-8a75ce002e3d admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:12.446060 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec0bfe5-9afc-42cc-a259-cad0f051951f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:12.446060 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0bfe5-9afc-42cc-a259-cad0f051951f admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:12.446060 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0bfe5-9afc-42cc-a259-cad0f051951f admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:12.446060 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0bfe5-9afc-42cc-a259-cad0f051951f admin admin] PATH_INFO: `/v3/projects/9c235ff08ff049e19dafd7c27df823e1/users/ffc92062ef424bec8c3da74d07fd6237/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:12.447624 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec0bfe5-9afc-42cc-a259-cad0f051951f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c235ff08ff049e19dafd7c27df823e1, user_id=ffc92062ef424bec8c3da74d07fd6237, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:12.450499 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec0bfe5-9afc-42cc-a259-cad0f051951f admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:12.465415 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1404/2812] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:24:12 2022] PUT /identity/v3/projects/9c235ff08ff049e19dafd7c27df823e1/users/ffc92062ef424bec8c3da74d07fd6237/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:24:12.538673 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60927c9c-9163-4b80-b42c-c5e0734dcc3f None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:12.539316 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60927c9c-9163-4b80-b42c-c5e0734dcc3f None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:12.540367 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60927c9c-9163-4b80-b42c-c5e0734dcc3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:12.580482 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-60927c9c-9163-4b80-b42c-c5e0734dcc3f None None] MFA Rules not processed for user `ffc92062ef424bec8c3da74d07fd6237`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:24:12.582805 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-60927c9c-9163-4b80-b42c-c5e0734dcc3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:12.745118 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1409/2813] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:24:12 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:24:12.755890 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec0bfe5-9afc-42cc-a259-cad0f051951f admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:12.758375 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-8ec0bfe5-9afc-42cc-a259-cad0f051951f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:12.872405 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec0bfe5-9afc-42cc-a259-cad0f051951f admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:12.876500 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf2a00d-74ad-425c-8b22-f83856a8cff4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:12.877509 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2a00d-74ad-425c-8b22-f83856a8cff4 service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:12.877891 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2a00d-74ad-425c-8b22-f83856a8cff4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:12.878266 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2a00d-74ad-425c-8b22-f83856a8cff4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:12.969361 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:24:12.972024 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf2a00d-74ad-425c-8b22-f83856a8cff4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:12.974428 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf2a00d-74ad-425c-8b22-f83856a8cff4 service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:12.990598 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1405/2814] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:24:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:24:13.183040 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f706be-d059-4416-b07b-7285725b97b3 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:13.183407 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f706be-d059-4416-b07b-7285725b97b3 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:13.183778 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f706be-d059-4416-b07b-7285725b97b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:13.246039 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-72f706be-d059-4416-b07b-7285725b97b3 None None] MFA Rules not processed for user `4d867759b4f249a393598964774a22fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:24:13.247829 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-72f706be-d059-4416-b07b-7285725b97b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:13.326012 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1410/2815] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:24:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:24:13.330411 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf2a00d-74ad-425c-8b22-f83856a8cff4 service cinder] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:13.332214 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-2cf2a00d-74ad-425c-8b22-f83856a8cff4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:13.434902 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf2a00d-74ad-425c-8b22-f83856a8cff4 service cinder] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:13.444998 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51838a14-a14d-4174-b714-04ea2fbf2d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:13.445693 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51838a14-a14d-4174-b714-04ea2fbf2d87 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:13.445978 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51838a14-a14d-4174-b714-04ea2fbf2d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:13.446257 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51838a14-a14d-4174-b714-04ea2fbf2d87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:13.446981 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51838a14-a14d-4174-b714-04ea2fbf2d87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:13.448933 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51838a14-a14d-4174-b714-04ea2fbf2d87 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:13.486331 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1406/2816] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:24:13 2022] POST /identity/v3/projects => generated 385 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:24:13.487202 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f706be-d059-4416-b07b-7285725b97b3 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:13.498775 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f706be-d059-4416-b07b-7285725b97b3 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:13.551023 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:24:13.557573 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8347a2-e89e-4247-a0c1-ba1d8a513e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:13.558255 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8347a2-e89e-4247-a0c1-ba1d8a513e55 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:13.558538 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8347a2-e89e-4247-a0c1-ba1d8a513e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:13.558814 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8347a2-e89e-4247-a0c1-ba1d8a513e55 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:13.559510 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8347a2-e89e-4247-a0c1-ba1d8a513e55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:13.561599 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8347a2-e89e-4247-a0c1-ba1d8a513e55 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:13.582915 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1411/2817] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:24:13 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 27 21:24:13.587209 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51838a14-a14d-4174-b714-04ea2fbf2d87 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:13.598021 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51838a14-a14d-4174-b714-04ea2fbf2d87 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:13.602981 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de8673a-9cad-4620-bdac-77e60445e24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:13.603682 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8673a-9cad-4620-bdac-77e60445e24d admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:13.603969 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8673a-9cad-4620-bdac-77e60445e24d admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:13.604242 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8673a-9cad-4620-bdac-77e60445e24d admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:13.604995 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de8673a-9cad-4620-bdac-77e60445e24d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:13.606668 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de8673a-9cad-4620-bdac-77e60445e24d admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:13.618944 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1407/2818] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:24:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:24:13.623366 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8347a2-e89e-4247-a0c1-ba1d8a513e55 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:13.634775 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8347a2-e89e-4247-a0c1-ba1d8a513e55 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:13.642287 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc0e9b-ec3d-45ba-ae25-10a1a064d45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:13.643277 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc0e9b-ec3d-45ba-ae25-10a1a064d45b admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:13.643462 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc0e9b-ec3d-45ba-ae25-10a1a064d45b admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:13.643831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc0e9b-ec3d-45ba-ae25-10a1a064d45b admin admin] PATH_INFO: `/v3/projects/1bc2225ec19243f9b09cc1847ef9bce7/users/a7d3c0ed840a4d559e1f0e227de8f024/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:13.665831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbc0e9b-ec3d-45ba-ae25-10a1a064d45b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bc2225ec19243f9b09cc1847ef9bce7, user_id=a7d3c0ed840a4d559e1f0e227de8f024, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:13.668580 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbc0e9b-ec3d-45ba-ae25-10a1a064d45b admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:13.684554 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1412/2819] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:24:13 2022] PUT /identity/v3/projects/1bc2225ec19243f9b09cc1847ef9bce7/users/a7d3c0ed840a4d559e1f0e227de8f024/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:24:13.688407 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de8673a-9cad-4620-bdac-77e60445e24d admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:13.701615 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de8673a-9cad-4620-bdac-77e60445e24d admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:13.766102 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc22a50-12d5-4c50-90c6-2400bb98888a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:13.767065 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc22a50-12d5-4c50-90c6-2400bb98888a admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:13.767374 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc22a50-12d5-4c50-90c6-2400bb98888a admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:13.767679 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc22a50-12d5-4c50-90c6-2400bb98888a admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:13.768490 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc22a50-12d5-4c50-90c6-2400bb98888a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:13.770446 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc22a50-12d5-4c50-90c6-2400bb98888a admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:13.785118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1408/2820] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:24:13 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:24:13.786382 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc0e9b-ec3d-45ba-ae25-10a1a064d45b admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:13.798057 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc0e9b-ec3d-45ba-ae25-10a1a064d45b admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:13.804360 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d5375-010d-4596-a9a7-244c438d3ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:13.805399 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d5375-010d-4596-a9a7-244c438d3ce5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:13.805807 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d5375-010d-4596-a9a7-244c438d3ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:13.806189 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2d5375-010d-4596-a9a7-244c438d3ce5 admin admin] PATH_INFO: `/v3/projects/1bc2225ec19243f9b09cc1847ef9bce7/users/a7d3c0ed840a4d559e1f0e227de8f024/roles/d336dd7b746c4488b33289c00302e4d5` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:13.808705 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2d5375-010d-4596-a9a7-244c438d3ce5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bc2225ec19243f9b09cc1847ef9bce7, user_id=a7d3c0ed840a4d559e1f0e227de8f024, role_id=d336dd7b746c4488b33289c00302e4d5)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:13.812197 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2d5375-010d-4596-a9a7-244c438d3ce5 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:13.824517 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1413/2821] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:24:13 2022] PUT /identity/v3/projects/1bc2225ec19243f9b09cc1847ef9bce7/users/a7d3c0ed840a4d559e1f0e227de8f024/roles/d336dd7b746c4488b33289c00302e4d5 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:24:14.648689 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc22a50-12d5-4c50-90c6-2400bb98888a admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:14.716989 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc22a50-12d5-4c50-90c6-2400bb98888a admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:14.740976 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:24:14.800102 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b87d5-7b28-419a-99d3-8ee75c426701 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:14.802008 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b87d5-7b28-419a-99d3-8ee75c426701 service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:14.802320 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b87d5-7b28-419a-99d3-8ee75c426701 service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:14.802690 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b87d5-7b28-419a-99d3-8ee75c426701 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:14.883961 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b87d5-7b28-419a-99d3-8ee75c426701 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:14.885507 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b87d5-7b28-419a-99d3-8ee75c426701 service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:14.897223 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1409/2822] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:24:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:24:15.647572 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d5375-010d-4596-a9a7-244c438d3ce5 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:15.664446 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2d5375-010d-4596-a9a7-244c438d3ce5 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:15.676958 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61914123-0ba0-4f54-8068-7fb22909b359 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:15.678032 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61914123-0ba0-4f54-8068-7fb22909b359 service cinder] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:15.678329 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61914123-0ba0-4f54-8068-7fb22909b359 service cinder] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:15.678612 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61914123-0ba0-4f54-8068-7fb22909b359 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:15.791954 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61914123-0ba0-4f54-8068-7fb22909b359 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:15.793702 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61914123-0ba0-4f54-8068-7fb22909b359 service cinder] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:15.809794 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1414/2823] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:24:15 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:24:18.861531 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86314743-3bfe-47a6-8932-e07888babc22 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:18.861531 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86314743-3bfe-47a6-8932-e07888babc22 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:18.861531 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86314743-3bfe-47a6-8932-e07888babc22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:18.905254 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-86314743-3bfe-47a6-8932-e07888babc22 None None] MFA Rules not processed for user `a7d3c0ed840a4d559e1f0e227de8f024`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:24:18.906896 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-86314743-3bfe-47a6-8932-e07888babc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:19.025579 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1410/2824] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:24:18 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:24:19.046040 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61914123-0ba0-4f54-8068-7fb22909b359 service cinder] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:19.047569 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-61914123-0ba0-4f54-8068-7fb22909b359 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:19.121633 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61914123-0ba0-4f54-8068-7fb22909b359 service cinder] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:19.125695 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0748bf3-3404-435c-afb8-c3c6118341ca service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '26cf962d151b4da7a82c33ef23b342e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:19.126387 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0748bf3-3404-435c-afb8-c3c6118341ca service glance] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:19.126676 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0748bf3-3404-435c-afb8-c3c6118341ca service glance] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:19.126942 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0748bf3-3404-435c-afb8-c3c6118341ca service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:19.202024 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0748bf3-3404-435c-afb8-c3c6118341ca service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:19.204172 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0748bf3-3404-435c-afb8-c3c6118341ca service glance] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:19.210704 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:24:19.220966 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1415/2825] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:24:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:24:19.339952 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a5907-9066-4420-b465-5beb6a413ef9 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:19.340228 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a5907-9066-4420-b465-5beb6a413ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:19.340578 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a5907-9066-4420-b465-5beb6a413ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:19.363294 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-c78a5907-9066-4420-b465-5beb6a413ef9 None None] MFA Rules not processed for user `f67110d591a94537a3df6a2d835370f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:24:19.364531 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-c78a5907-9066-4420-b465-5beb6a413ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:19.424655 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1411/2826] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Jun 27 21:24:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:24:19.439198 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0748bf3-3404-435c-afb8-c3c6118341ca service glance] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:19.440768 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-e0748bf3-3404-435c-afb8-c3c6118341ca service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:19.516548 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0748bf3-3404-435c-afb8-c3c6118341ca service glance] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:19.522233 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6141e58e-f730-4bca-b5eb-8ff8ad4132f4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bed6b231ee647d8a350da055d8e59d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:19.522887 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141e58e-f730-4bca-b5eb-8ff8ad4132f4 service swift] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:19.523170 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141e58e-f730-4bca-b5eb-8ff8ad4132f4 service swift] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:19.523438 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141e58e-f730-4bca-b5eb-8ff8ad4132f4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:19.532507 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6141e58e-f730-4bca-b5eb-8ff8ad4132f4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:19.534154 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6141e58e-f730-4bca-b5eb-8ff8ad4132f4 service swift] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:19.542801 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1416/2827] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Jun 27 21:24:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 21:24:20.275895 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a5907-9066-4420-b465-5beb6a413ef9 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:20.285787 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=100 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:24:20.370918 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a5907-9066-4420-b465-5beb6a413ef9 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:20.375967 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f3bb67-e6c2-4827-ae22-d6c2325e3878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:20.377004 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3bb67-e6c2-4827-ae22-d6c2325e3878 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:20.377352 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3bb67-e6c2-4827-ae22-d6c2325e3878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:20.377808 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3bb67-e6c2-4827-ae22-d6c2325e3878 admin admin] PATH_INFO: `/v3/users/eec869d87067467d81611326c4f773ed` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:20.379199 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f3bb67-e6c2-4827-ae22-d6c2325e3878 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eec869d87067467d81611326c4f773ed)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:20.381480 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f3bb67-e6c2-4827-ae22-d6c2325e3878 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:20.461420 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-f7f3bb67-e6c2-4827-ae22-d6c2325e3878 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eec869d87067467d81611326c4f773ed'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:20.471943 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-f7f3bb67-e6c2-4827-ae22-d6c2325e3878 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eec869d87067467d81611326c4f773ed'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:20.472667 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-f7f3bb67-e6c2-4827-ae22-d6c2325e3878 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eec869d87067467d81611326c4f773ed'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:20.473296 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-f7f3bb67-e6c2-4827-ae22-d6c2325e3878 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eec869d87067467d81611326c4f773ed'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:20.486079 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-f7f3bb67-e6c2-4827-ae22-d6c2325e3878 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eec869d87067467d81611326c4f773ed'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:20.514502 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1412/2828] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:24:20 2022] DELETE /identity/v3/users/eec869d87067467d81611326c4f773ed => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:24:20.518937 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac255a9-6567-4f97-8287-d5ea9fe083d8 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:20.519451 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac255a9-6567-4f97-8287-d5ea9fe083d8 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:20.519857 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac255a9-6567-4f97-8287-d5ea9fe083d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:20.557543 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-aac255a9-6567-4f97-8287-d5ea9fe083d8 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:24:20.559133 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-aac255a9-6567-4f97-8287-d5ea9fe083d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:20.637625 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1417/2829] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:24:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:24:20.647895 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f3bb67-e6c2-4827-ae22-d6c2325e3878 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:20.649447 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-f7f3bb67-e6c2-4827-ae22-d6c2325e3878 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:20.756204 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f3bb67-e6c2-4827-ae22-d6c2325e3878 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:20.761437 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4251b-4d29-48a9-be76-9bca0373c7f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:20.762455 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4251b-4d29-48a9-be76-9bca0373c7f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:20.762938 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4251b-4d29-48a9-be76-9bca0373c7f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:20.763381 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a4251b-4d29-48a9-be76-9bca0373c7f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:20.780716 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a4251b-4d29-48a9-be76-9bca0373c7f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:20.783923 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a4251b-4d29-48a9-be76-9bca0373c7f1 service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:20.798953 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1413/2830] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:24:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:24:21.167528 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac255a9-6567-4f97-8287-d5ea9fe083d8 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:21.169177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-aac255a9-6567-4f97-8287-d5ea9fe083d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:21.183075 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac255a9-6567-4f97-8287-d5ea9fe083d8 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:21.188395 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675853f5-abf6-4f96-acb5-4b8a1f322c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:21.189387 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675853f5-abf6-4f96-acb5-4b8a1f322c05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:21.189674 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675853f5-abf6-4f96-acb5-4b8a1f322c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:21.189963 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675853f5-abf6-4f96-acb5-4b8a1f322c05 admin admin] PATH_INFO: `/v3/projects/8e903efc6e8f487d950e079eff67922c` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:21.192562 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675853f5-abf6-4f96-acb5-4b8a1f322c05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e903efc6e8f487d950e079eff67922c)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:21.194839 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675853f5-abf6-4f96-acb5-4b8a1f322c05 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:21.269313 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-675853f5-abf6-4f96-acb5-4b8a1f322c05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e903efc6e8f487d950e079eff67922c', 'request_id': 'req-675853f5-abf6-4f96-acb5-4b8a1f322c05'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:21.279559 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-675853f5-abf6-4f96-acb5-4b8a1f322c05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e903efc6e8f487d950e079eff67922c', 'request_id': 'req-675853f5-abf6-4f96-acb5-4b8a1f322c05'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:21.289457 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-675853f5-abf6-4f96-acb5-4b8a1f322c05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e903efc6e8f487d950e079eff67922c', 'request_id': 'req-675853f5-abf6-4f96-acb5-4b8a1f322c05'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:21.301110 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-675853f5-abf6-4f96-acb5-4b8a1f322c05 admin admin] The token cache is being invalidate because project 8e903efc6e8f487d950e079eff67922c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:24:21.301433 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-675853f5-abf6-4f96-acb5-4b8a1f322c05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:21.302040 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-675853f5-abf6-4f96-acb5-4b8a1f322c05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:21.303928 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1418/2831] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:24:21 2022] DELETE /identity/v3/projects/8e903efc6e8f487d950e079eff67922c => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:24:21.309789 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4251b-4d29-48a9-be76-9bca0373c7f1 service neutron] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:21.322596 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a4251b-4d29-48a9-be76-9bca0373c7f1 service neutron] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:21.363225 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:24:21.417982 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c321cd06-7646-430a-a5ad-7ec3faa6ad2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:21.419179 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321cd06-7646-430a-a5ad-7ec3faa6ad2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:21.419651 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321cd06-7646-430a-a5ad-7ec3faa6ad2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:21.420079 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c321cd06-7646-430a-a5ad-7ec3faa6ad2a admin admin] PATH_INFO: `/v3/users/3e7b5d6f8b71481384436c953a3775a5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:21.422057 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c321cd06-7646-430a-a5ad-7ec3faa6ad2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e7b5d6f8b71481384436c953a3775a5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:21.424274 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c321cd06-7646-430a-a5ad-7ec3faa6ad2a admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:21.505284 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c321cd06-7646-430a-a5ad-7ec3faa6ad2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e7b5d6f8b71481384436c953a3775a5'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:21.521995 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c321cd06-7646-430a-a5ad-7ec3faa6ad2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e7b5d6f8b71481384436c953a3775a5'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:21.524029 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c321cd06-7646-430a-a5ad-7ec3faa6ad2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e7b5d6f8b71481384436c953a3775a5'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:21.524875 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c321cd06-7646-430a-a5ad-7ec3faa6ad2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e7b5d6f8b71481384436c953a3775a5'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:21.542269 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c321cd06-7646-430a-a5ad-7ec3faa6ad2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e7b5d6f8b71481384436c953a3775a5'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:21.581498 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1414/2832] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:24:21 2022] DELETE /identity/v3/users/3e7b5d6f8b71481384436c953a3775a5 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:24:22.147584 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675853f5-abf6-4f96-acb5-4b8a1f322c05 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:22.149949 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-675853f5-abf6-4f96-acb5-4b8a1f322c05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:22.183794 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675853f5-abf6-4f96-acb5-4b8a1f322c05 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:22.215770 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:24:22.318638 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ac303a-d5f9-4555-8c38-d6af2c5f36e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:22.319725 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac303a-d5f9-4555-8c38-d6af2c5f36e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:22.320006 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac303a-d5f9-4555-8c38-d6af2c5f36e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:22.320282 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac303a-d5f9-4555-8c38-d6af2c5f36e9 admin admin] PATH_INFO: `/v3/projects/935a2203fe1a4234963a18b6a048f163` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:22.324493 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ac303a-d5f9-4555-8c38-d6af2c5f36e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=935a2203fe1a4234963a18b6a048f163)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:22.327781 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ac303a-d5f9-4555-8c38-d6af2c5f36e9 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:22.456403 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-62ac303a-d5f9-4555-8c38-d6af2c5f36e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '935a2203fe1a4234963a18b6a048f163', 'request_id': 'req-62ac303a-d5f9-4555-8c38-d6af2c5f36e9'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:22.471250 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-62ac303a-d5f9-4555-8c38-d6af2c5f36e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '935a2203fe1a4234963a18b6a048f163', 'request_id': 'req-62ac303a-d5f9-4555-8c38-d6af2c5f36e9'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:22.492980 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-62ac303a-d5f9-4555-8c38-d6af2c5f36e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '935a2203fe1a4234963a18b6a048f163', 'request_id': 'req-62ac303a-d5f9-4555-8c38-d6af2c5f36e9'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:22.514411 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-62ac303a-d5f9-4555-8c38-d6af2c5f36e9 admin admin] The token cache is being invalidate because project 935a2203fe1a4234963a18b6a048f163 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:24:22.515197 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-62ac303a-d5f9-4555-8c38-d6af2c5f36e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:22.516155 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-62ac303a-d5f9-4555-8c38-d6af2c5f36e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:22.521329 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1419/2833] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:24:22 2022] DELETE /identity/v3/projects/935a2203fe1a4234963a18b6a048f163 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:24:22.526830 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c321cd06-7646-430a-a5ad-7ec3faa6ad2a admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:22.528293 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-c321cd06-7646-430a-a5ad-7ec3faa6ad2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:22.555426 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c321cd06-7646-430a-a5ad-7ec3faa6ad2a admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:22.636792 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0756ee9e-c61c-4a82-aa78-3b9858595159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:22.637749 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756ee9e-c61c-4a82-aa78-3b9858595159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:22.638034 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756ee9e-c61c-4a82-aa78-3b9858595159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:22.638308 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756ee9e-c61c-4a82-aa78-3b9858595159 admin admin] PATH_INFO: `/v3/users/ffc92062ef424bec8c3da74d07fd6237` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:22.639547 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0756ee9e-c61c-4a82-aa78-3b9858595159 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffc92062ef424bec8c3da74d07fd6237)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:22.641622 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0756ee9e-c61c-4a82-aa78-3b9858595159 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:22.720068 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-0756ee9e-c61c-4a82-aa78-3b9858595159 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffc92062ef424bec8c3da74d07fd6237'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:22.722397 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:24:22.735633 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-0756ee9e-c61c-4a82-aa78-3b9858595159 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffc92062ef424bec8c3da74d07fd6237'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:22.737870 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-0756ee9e-c61c-4a82-aa78-3b9858595159 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffc92062ef424bec8c3da74d07fd6237'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:22.739322 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-0756ee9e-c61c-4a82-aa78-3b9858595159 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffc92062ef424bec8c3da74d07fd6237'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:22.752772 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-0756ee9e-c61c-4a82-aa78-3b9858595159 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffc92062ef424bec8c3da74d07fd6237'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:22.777632 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1415/2834] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:24:22 2022] DELETE /identity/v3/users/ffc92062ef424bec8c3da74d07fd6237 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:24:23.189855 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ac303a-d5f9-4555-8c38-d6af2c5f36e9 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:23.194773 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-62ac303a-d5f9-4555-8c38-d6af2c5f36e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:23.249803 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ac303a-d5f9-4555-8c38-d6af2c5f36e9 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:23.334253 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26709ad-a41f-4f14-b524-7c6c9842b959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:23.335603 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26709ad-a41f-4f14-b524-7c6c9842b959 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:23.336145 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26709ad-a41f-4f14-b524-7c6c9842b959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:23.336634 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26709ad-a41f-4f14-b524-7c6c9842b959 admin admin] PATH_INFO: `/v3/projects/9c235ff08ff049e19dafd7c27df823e1` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:23.338380 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26709ad-a41f-4f14-b524-7c6c9842b959 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c235ff08ff049e19dafd7c27df823e1)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:23.340427 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26709ad-a41f-4f14-b524-7c6c9842b959 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:23.406790 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:24:23.412120 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-d26709ad-a41f-4f14-b524-7c6c9842b959 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c235ff08ff049e19dafd7c27df823e1', 'request_id': 'req-d26709ad-a41f-4f14-b524-7c6c9842b959'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:23.433473 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-d26709ad-a41f-4f14-b524-7c6c9842b959 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c235ff08ff049e19dafd7c27df823e1', 'request_id': 'req-d26709ad-a41f-4f14-b524-7c6c9842b959'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:23.481829 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-d26709ad-a41f-4f14-b524-7c6c9842b959 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c235ff08ff049e19dafd7c27df823e1', 'request_id': 'req-d26709ad-a41f-4f14-b524-7c6c9842b959'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:23.494203 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-d26709ad-a41f-4f14-b524-7c6c9842b959 admin admin] The token cache is being invalidate because project 9c235ff08ff049e19dafd7c27df823e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:24:23.494502 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-d26709ad-a41f-4f14-b524-7c6c9842b959 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:23.495166 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-d26709ad-a41f-4f14-b524-7c6c9842b959 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:23.497023 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1420/2835] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:24:23 2022] DELETE /identity/v3/projects/9c235ff08ff049e19dafd7c27df823e1 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:24:29.673036 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d82db-c59d-440b-8be1-caeeb5b6f2f7 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:29.673610 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d82db-c59d-440b-8be1-caeeb5b6f2f7 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:29.673610 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d82db-c59d-440b-8be1-caeeb5b6f2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:29.690805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-7a2d82db-c59d-440b-8be1-caeeb5b6f2f7 None None] MFA Rules not processed for user `4d867759b4f249a393598964774a22fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:24:29.692022 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-7a2d82db-c59d-440b-8be1-caeeb5b6f2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:29.756842 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1416/2836] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:24:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:24:30.854878 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26709ad-a41f-4f14-b524-7c6c9842b959 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:30.856434 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d26709ad-a41f-4f14-b524-7c6c9842b959 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:30.931747 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26709ad-a41f-4f14-b524-7c6c9842b959 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:30.937302 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d2693a-435e-4ca7-b4af-265ce1ba9de1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0437c45d72cd4a38b9d48ee6f50946fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:30.937978 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d2693a-435e-4ca7-b4af-265ce1ba9de1 service placement] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:30.938252 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d2693a-435e-4ca7-b4af-265ce1ba9de1 service placement] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:30.938529 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d2693a-435e-4ca7-b4af-265ce1ba9de1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:30.940331 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-79d2693a-435e-4ca7-b4af-265ce1ba9de1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:30.952761 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d2693a-435e-4ca7-b4af-265ce1ba9de1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:30.954419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d2693a-435e-4ca7-b4af-265ce1ba9de1 service placement] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:30.966126 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1421/2837] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:24:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:24:31.792366 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2d82db-c59d-440b-8be1-caeeb5b6f2f7 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:31.794790 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-7a2d82db-c59d-440b-8be1-caeeb5b6f2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:31.873027 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2d82db-c59d-440b-8be1-caeeb5b6f2f7 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:31.880760 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de34987-82db-4e27-a05a-7f217e0c6498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:31.881807 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de34987-82db-4e27-a05a-7f217e0c6498 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:31.881958 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de34987-82db-4e27-a05a-7f217e0c6498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:31.882177 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de34987-82db-4e27-a05a-7f217e0c6498 admin admin] PATH_INFO: `/v3/users/2619c847e3f24286a60108a316f93a22` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:31.883435 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de34987-82db-4e27-a05a-7f217e0c6498 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2619c847e3f24286a60108a316f93a22)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:31.885430 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de34987-82db-4e27-a05a-7f217e0c6498 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:31.922276 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2d671-d4ff-4c4f-8491-e599fec3d19a None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:31.922843 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2d671-d4ff-4c4f-8491-e599fec3d19a None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:31.922843 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2d671-d4ff-4c4f-8491-e599fec3d19a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:31.942271 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-2de34987-82db-4e27-a05a-7f217e0c6498 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2619c847e3f24286a60108a316f93a22'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:31.949177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-a8d2d671-d4ff-4c4f-8491-e599fec3d19a None None] MFA Rules not processed for user `f67110d591a94537a3df6a2d835370f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:24:31.950261 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a8d2d671-d4ff-4c4f-8491-e599fec3d19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:31.951258 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-2de34987-82db-4e27-a05a-7f217e0c6498 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2619c847e3f24286a60108a316f93a22'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:31.952186 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-2de34987-82db-4e27-a05a-7f217e0c6498 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2619c847e3f24286a60108a316f93a22'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:31.952795 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-2de34987-82db-4e27-a05a-7f217e0c6498 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2619c847e3f24286a60108a316f93a22'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:31.963770 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-2de34987-82db-4e27-a05a-7f217e0c6498 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2619c847e3f24286a60108a316f93a22'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:31.973611 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:24:31.992391 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1417/2838] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:24:31 2022] DELETE /identity/v3/users/2619c847e3f24286a60108a316f93a22 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:24:31.996959 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ae9c2-8a1a-41bd-a77d-cae231f2bec4 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:31.997227 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ae9c2-8a1a-41bd-a77d-cae231f2bec4 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:31.997641 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ae9c2-8a1a-41bd-a77d-cae231f2bec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:32.022377 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-416ae9c2-8a1a-41bd-a77d-cae231f2bec4 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:24:32.024667 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-416ae9c2-8a1a-41bd-a77d-cae231f2bec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:32.034680 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1422/2839] 10.222.0.13 () {58 vars in 1009 bytes} [Mon Jun 27 21:24:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:24:32.044539 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d2d671-d4ff-4c4f-8491-e599fec3d19a None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:32.046695 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-a8d2d671-d4ff-4c4f-8491-e599fec3d19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:32.094533 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1418/2840] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:24:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:24:32.105226 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ae9c2-8a1a-41bd-a77d-cae231f2bec4 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:32.106933 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-416ae9c2-8a1a-41bd-a77d-cae231f2bec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:32.131858 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d2d671-d4ff-4c4f-8491-e599fec3d19a None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:32.136900 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:24:32.180589 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ae9c2-8a1a-41bd-a77d-cae231f2bec4 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:32.185434 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec091822-e317-4e8f-a003-1223f16379de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:32.186133 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec091822-e317-4e8f-a003-1223f16379de service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:32.186413 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec091822-e317-4e8f-a003-1223f16379de service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:32.186725 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec091822-e317-4e8f-a003-1223f16379de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:32.198672 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec091822-e317-4e8f-a003-1223f16379de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:32.200310 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec091822-e317-4e8f-a003-1223f16379de service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:32.201419 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd67093-cfb5-4e2c-88b9-c10c28892d38 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bed6b231ee647d8a350da055d8e59d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:32.202095 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd67093-cfb5-4e2c-88b9-c10c28892d38 service swift] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:32.202378 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd67093-cfb5-4e2c-88b9-c10c28892d38 service swift] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:32.202686 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd67093-cfb5-4e2c-88b9-c10c28892d38 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:32.211849 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1419/2841] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:24:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:24:32.273920 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd67093-cfb5-4e2c-88b9-c10c28892d38 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:32.276054 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd67093-cfb5-4e2c-88b9-c10c28892d38 service swift] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:32.287825 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1423/2842] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Jun 27 21:24:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 21:24:32.847456 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec091822-e317-4e8f-a003-1223f16379de service neutron] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:32.849251 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-ec091822-e317-4e8f-a003-1223f16379de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:32.885384 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec091822-e317-4e8f-a003-1223f16379de service neutron] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:32.887945 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2071fc8-d7c3-46fa-b011-1844931ee778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:32.889007 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2071fc8-d7c3-46fa-b011-1844931ee778 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:32.889396 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2071fc8-d7c3-46fa-b011-1844931ee778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:32.890016 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2071fc8-d7c3-46fa-b011-1844931ee778 admin admin] PATH_INFO: `/v3/projects/98d7f881222c45ada5de9a6170e69954` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:32.896888 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2071fc8-d7c3-46fa-b011-1844931ee778 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98d7f881222c45ada5de9a6170e69954)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:32.899229 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2071fc8-d7c3-46fa-b011-1844931ee778 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:33.016129 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-e2071fc8-d7c3-46fa-b011-1844931ee778 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98d7f881222c45ada5de9a6170e69954', 'request_id': 'req-e2071fc8-d7c3-46fa-b011-1844931ee778'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:33.026684 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-e2071fc8-d7c3-46fa-b011-1844931ee778 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98d7f881222c45ada5de9a6170e69954', 'request_id': 'req-e2071fc8-d7c3-46fa-b011-1844931ee778'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:33.037629 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-e2071fc8-d7c3-46fa-b011-1844931ee778 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98d7f881222c45ada5de9a6170e69954', 'request_id': 'req-e2071fc8-d7c3-46fa-b011-1844931ee778'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:33.055314 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-e2071fc8-d7c3-46fa-b011-1844931ee778 admin admin] The token cache is being invalidate because project 98d7f881222c45ada5de9a6170e69954 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:24:33.055819 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-e2071fc8-d7c3-46fa-b011-1844931ee778 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:33.056979 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-e2071fc8-d7c3-46fa-b011-1844931ee778 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:33.059447 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1420/2843] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:24:32 2022] DELETE /identity/v3/projects/98d7f881222c45ada5de9a6170e69954 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:24:33.065465 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef43fdd-f198-4d3e-887d-407efbe8b744 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:33.066324 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef43fdd-f198-4d3e-887d-407efbe8b744 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:33.066324 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef43fdd-f198-4d3e-887d-407efbe8b744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:33.110601 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-cef43fdd-f198-4d3e-887d-407efbe8b744 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:24:33.111910 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-cef43fdd-f198-4d3e-887d-407efbe8b744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:33.277983 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1424/2844] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:24:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:24:33.286065 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a62cb-c9c4-4968-955c-3530546dcafa None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:33.286772 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a62cb-c9c4-4968-955c-3530546dcafa None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:33.287189 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a62cb-c9c4-4968-955c-3530546dcafa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:33.312118 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-1c8a62cb-c9c4-4968-955c-3530546dcafa None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:24:33.314491 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-1c8a62cb-c9c4-4968-955c-3530546dcafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:33.330616 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1421/2845] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:24:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:24:33.331554 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef43fdd-f198-4d3e-887d-407efbe8b744 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:33.352986 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef43fdd-f198-4d3e-887d-407efbe8b744 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:33.358233 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5946c08-76ba-47a6-a0d2-bf0b7f742118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:33.358986 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5946c08-76ba-47a6-a0d2-bf0b7f742118 admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:33.359351 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5946c08-76ba-47a6-a0d2-bf0b7f742118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:33.359631 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5946c08-76ba-47a6-a0d2-bf0b7f742118 admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:33.360530 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5946c08-76ba-47a6-a0d2-bf0b7f742118 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:24:33.360906 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5946c08-76ba-47a6-a0d2-bf0b7f742118 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:33.362710 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5946c08-76ba-47a6-a0d2-bf0b7f742118 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:33.387703 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1425/2846] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 27 21:24:33.394294 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a62cb-c9c4-4968-955c-3530546dcafa None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:33.409692 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a62cb-c9c4-4968-955c-3530546dcafa None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:33.416117 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981a0e9-cd20-4b03-ae1c-ae8afd38c1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:33.416898 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981a0e9-cd20-4b03-ae1c-ae8afd38c1f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:33.418269 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981a0e9-cd20-4b03-ae1c-ae8afd38c1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:33.418618 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3981a0e9-cd20-4b03-ae1c-ae8afd38c1f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:33.419400 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3981a0e9-cd20-4b03-ae1c-ae8afd38c1f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:33.422425 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3981a0e9-cd20-4b03-ae1c-ae8afd38c1f6 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:33.444246 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:24:33.484142 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1422/2847] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:24:33 2022] POST /identity/v3/projects => generated 373 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:24:33.487871 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5946c08-76ba-47a6-a0d2-bf0b7f742118 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:33.504741 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5946c08-76ba-47a6-a0d2-bf0b7f742118 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:33.564716 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:24:33.595814 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e6c52-776b-4e6b-89b9-c20fbb37063b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:33.597254 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e6c52-776b-4e6b-89b9-c20fbb37063b admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:33.597254 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e6c52-776b-4e6b-89b9-c20fbb37063b admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:33.597757 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e6c52-776b-4e6b-89b9-c20fbb37063b admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:33.598639 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e6c52-776b-4e6b-89b9-c20fbb37063b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:33.602394 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e6c52-776b-4e6b-89b9-c20fbb37063b admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:33.639960 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1426/2848] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:24:33 2022] POST /identity/v3/users => generated 287 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:24:33.641895 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981a0e9-cd20-4b03-ae1c-ae8afd38c1f6 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:33.671551 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3981a0e9-cd20-4b03-ae1c-ae8afd38c1f6 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:33.710839 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e465ac-c1d0-4186-b6cc-77a0ecc1b658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:33.711971 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e465ac-c1d0-4186-b6cc-77a0ecc1b658 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:33.712375 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e465ac-c1d0-4186-b6cc-77a0ecc1b658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:33.712748 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e465ac-c1d0-4186-b6cc-77a0ecc1b658 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:33.713649 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e465ac-c1d0-4186-b6cc-77a0ecc1b658 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:33.715420 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e465ac-c1d0-4186-b6cc-77a0ecc1b658 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:33.748211 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1423/2849] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:24:33.752020 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e6c52-776b-4e6b-89b9-c20fbb37063b admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:33.783902 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e6c52-776b-4e6b-89b9-c20fbb37063b admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:33.792125 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b8938d-b598-4ced-b536-77c7d30a64de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:33.793937 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b8938d-b598-4ced-b536-77c7d30a64de admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:33.794391 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b8938d-b598-4ced-b536-77c7d30a64de admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:33.794828 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b8938d-b598-4ced-b536-77c7d30a64de admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:33.806771 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b8938d-b598-4ced-b536-77c7d30a64de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:33.808681 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b8938d-b598-4ced-b536-77c7d30a64de admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:33.833738 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1427/2850] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 27 21:24:33.837736 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e465ac-c1d0-4186-b6cc-77a0ecc1b658 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:33.866903 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e465ac-c1d0-4186-b6cc-77a0ecc1b658 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:33.894696 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d533da1-ea09-4021-b1d7-c503bc6e9868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:33.903919 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d533da1-ea09-4021-b1d7-c503bc6e9868 admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:33.904216 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d533da1-ea09-4021-b1d7-c503bc6e9868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:33.904487 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d533da1-ea09-4021-b1d7-c503bc6e9868 admin admin] PATH_INFO: `/v3/projects/421af615d28c43c19d789a9e5189e5ab/users/54eb5e8ca50c4dd6b4b771dc5db50e20/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:33.953397 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d533da1-ea09-4021-b1d7-c503bc6e9868 admin admin] RBAC: Authorizing `identity:create_grant(project_id=421af615d28c43c19d789a9e5189e5ab, user_id=54eb5e8ca50c4dd6b4b771dc5db50e20, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:33.956468 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d533da1-ea09-4021-b1d7-c503bc6e9868 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:34.006406 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1424/2851] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:24:33 2022] PUT /identity/v3/projects/421af615d28c43c19d789a9e5189e5ab/users/54eb5e8ca50c4dd6b4b771dc5db50e20/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:24:34.019406 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe6d5e-de31-4b4d-90ea-7027ee083acd None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:34.023324 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe6d5e-de31-4b4d-90ea-7027ee083acd None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:34.023512 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe6d5e-de31-4b4d-90ea-7027ee083acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:34.102630 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-7ebe6d5e-de31-4b4d-90ea-7027ee083acd None None] MFA Rules not processed for user `54eb5e8ca50c4dd6b4b771dc5db50e20`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:24:34.105637 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-7ebe6d5e-de31-4b4d-90ea-7027ee083acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:34.279766 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1428/2852] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:24:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:24:34.292737 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d533da1-ea09-4021-b1d7-c503bc6e9868 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:34.296513 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-3d533da1-ea09-4021-b1d7-c503bc6e9868 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:34.420863 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d533da1-ea09-4021-b1d7-c503bc6e9868 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:34.425790 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2fe36e-3c60-4ed8-8b9c-0b70036893d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:34.426558 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2fe36e-3c60-4ed8-8b9c-0b70036893d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:34.426935 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2fe36e-3c60-4ed8-8b9c-0b70036893d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:34.427609 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2fe36e-3c60-4ed8-8b9c-0b70036893d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:34.439809 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2fe36e-3c60-4ed8-8b9c-0b70036893d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:34.442280 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2fe36e-3c60-4ed8-8b9c-0b70036893d1 service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:34.468756 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1425/2853] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:24:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:24:34.491000 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebe6d5e-de31-4b4d-90ea-7027ee083acd None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:34.590338 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebe6d5e-de31-4b4d-90ea-7027ee083acd None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:34.595908 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659473e8-39c2-4134-ac8d-e25ab058db9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:34.597469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659473e8-39c2-4134-ac8d-e25ab058db9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:34.598004 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659473e8-39c2-4134-ac8d-e25ab058db9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:34.598519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659473e8-39c2-4134-ac8d-e25ab058db9e admin admin] PATH_INFO: `/v3/users/54eb5e8ca50c4dd6b4b771dc5db50e20` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:34.600098 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659473e8-39c2-4134-ac8d-e25ab058db9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=54eb5e8ca50c4dd6b4b771dc5db50e20)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:34.602288 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659473e8-39c2-4134-ac8d-e25ab058db9e admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:34.656882 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-659473e8-39c2-4134-ac8d-e25ab058db9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54eb5e8ca50c4dd6b4b771dc5db50e20'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:34.660002 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:24:34.666449 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-659473e8-39c2-4134-ac8d-e25ab058db9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54eb5e8ca50c4dd6b4b771dc5db50e20'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:34.667415 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-659473e8-39c2-4134-ac8d-e25ab058db9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54eb5e8ca50c4dd6b4b771dc5db50e20'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:34.668360 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-659473e8-39c2-4134-ac8d-e25ab058db9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54eb5e8ca50c4dd6b4b771dc5db50e20'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:34.676425 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-659473e8-39c2-4134-ac8d-e25ab058db9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54eb5e8ca50c4dd6b4b771dc5db50e20'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:34.707061 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1429/2854] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:24:34 2022] DELETE /identity/v3/users/54eb5e8ca50c4dd6b4b771dc5db50e20 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:24:34.712685 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283a63f-5f7a-4ae3-85b3-73105bbef96e None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:34.713060 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283a63f-5f7a-4ae3-85b3-73105bbef96e None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:34.713371 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283a63f-5f7a-4ae3-85b3-73105bbef96e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:34.736610 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-3283a63f-5f7a-4ae3-85b3-73105bbef96e None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:24:34.738195 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-3283a63f-5f7a-4ae3-85b3-73105bbef96e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:34.807747 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1426/2855] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:24:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:24:34.819026 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659473e8-39c2-4134-ac8d-e25ab058db9e admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:34.821074 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-659473e8-39c2-4134-ac8d-e25ab058db9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:34.890450 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659473e8-39c2-4134-ac8d-e25ab058db9e admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:34.895056 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9040c5-c7af-4935-9548-1e8b973efc8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:34.895738 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9040c5-c7af-4935-9548-1e8b973efc8a service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:34.896092 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9040c5-c7af-4935-9548-1e8b973efc8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:34.897461 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9040c5-c7af-4935-9548-1e8b973efc8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:34.908292 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9040c5-c7af-4935-9548-1e8b973efc8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:34.909858 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9040c5-c7af-4935-9548-1e8b973efc8a service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:34.921831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1430/2856] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:24:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:24:35.247119 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3283a63f-5f7a-4ae3-85b3-73105bbef96e None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:35.248713 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-3283a63f-5f7a-4ae3-85b3-73105bbef96e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:35.265542 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3283a63f-5f7a-4ae3-85b3-73105bbef96e None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:35.271000 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0799518-761e-4990-95c5-5125fa6fbf08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:35.272682 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0799518-761e-4990-95c5-5125fa6fbf08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:35.272682 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0799518-761e-4990-95c5-5125fa6fbf08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:35.274599 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0799518-761e-4990-95c5-5125fa6fbf08 admin admin] PATH_INFO: `/v3/projects/421af615d28c43c19d789a9e5189e5ab` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:35.276395 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0799518-761e-4990-95c5-5125fa6fbf08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=421af615d28c43c19d789a9e5189e5ab)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:35.278585 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0799518-761e-4990-95c5-5125fa6fbf08 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:35.342279 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c0799518-761e-4990-95c5-5125fa6fbf08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '421af615d28c43c19d789a9e5189e5ab', 'request_id': 'req-c0799518-761e-4990-95c5-5125fa6fbf08'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:35.345218 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:24:35.354668 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c0799518-761e-4990-95c5-5125fa6fbf08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '421af615d28c43c19d789a9e5189e5ab', 'request_id': 'req-c0799518-761e-4990-95c5-5125fa6fbf08'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:35.371952 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c0799518-761e-4990-95c5-5125fa6fbf08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '421af615d28c43c19d789a9e5189e5ab', 'request_id': 'req-c0799518-761e-4990-95c5-5125fa6fbf08'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:35.386043 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c0799518-761e-4990-95c5-5125fa6fbf08 admin admin] The token cache is being invalidate because project 421af615d28c43c19d789a9e5189e5ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:24:35.386361 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c0799518-761e-4990-95c5-5125fa6fbf08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:35.387260 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-c0799518-761e-4990-95c5-5125fa6fbf08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:24:35.390169 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1427/2857] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:24:35 2022] DELETE /identity/v3/projects/421af615d28c43c19d789a9e5189e5ab => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:24:35.396501 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42011c57-c3ac-4f73-89bc-c5e4963979ac None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:35.397177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42011c57-c3ac-4f73-89bc-c5e4963979ac None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:35.397265 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42011c57-c3ac-4f73-89bc-c5e4963979ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:35.422524 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-42011c57-c3ac-4f73-89bc-c5e4963979ac None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:24:35.424026 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-42011c57-c3ac-4f73-89bc-c5e4963979ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:35.531873 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1431/2858] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:24:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:24:35.533584 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64223c26-0697-43a1-8dda-a7c6ba84fb3d None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:35.533969 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64223c26-0697-43a1-8dda-a7c6ba84fb3d None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:35.534355 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64223c26-0697-43a1-8dda-a7c6ba84fb3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:35.554069 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-64223c26-0697-43a1-8dda-a7c6ba84fb3d None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:24:35.555313 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-64223c26-0697-43a1-8dda-a7c6ba84fb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:35.564312 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1428/2859] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:24:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:24:35.569206 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42011c57-c3ac-4f73-89bc-c5e4963979ac None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:35.579778 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42011c57-c3ac-4f73-89bc-c5e4963979ac None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:35.601701 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bd4ea2-7b40-4cf5-88a1-c588c076d2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:35.602349 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd4ea2-7b40-4cf5-88a1-c588c076d2fb admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:35.602629 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd4ea2-7b40-4cf5-88a1-c588c076d2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:35.602902 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd4ea2-7b40-4cf5-88a1-c588c076d2fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:35.603652 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bd4ea2-7b40-4cf5-88a1-c588c076d2fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80388) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 27 21:24:35.603958 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bd4ea2-7b40-4cf5-88a1-c588c076d2fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:35.605637 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bd4ea2-7b40-4cf5-88a1-c588c076d2fb admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:35.618629 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1432/2860] 10.222.0.13 () {58 vars in 1152 bytes} [Mon Jun 27 21:24:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 27 21:24:35.622835 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64223c26-0697-43a1-8dda-a7c6ba84fb3d None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:35.633002 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64223c26-0697-43a1-8dda-a7c6ba84fb3d None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:35.638603 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ec5ba-9d4b-4bb3-b00c-595ebea8ec0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:35.639294 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ec5ba-9d4b-4bb3-b00c-595ebea8ec0c admin admin] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:35.639580 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ec5ba-9d4b-4bb3-b00c-595ebea8ec0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:35.639858 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ec5ba-9d4b-4bb3-b00c-595ebea8ec0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:35.640556 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884ec5ba-9d4b-4bb3-b00c-595ebea8ec0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:35.642508 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884ec5ba-9d4b-4bb3-b00c-595ebea8ec0c admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:35.674725 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1429/2861] 10.222.0.13 () {60 vars in 1152 bytes} [Mon Jun 27 21:24:35 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:24:35.679018 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bd4ea2-7b40-4cf5-88a1-c588c076d2fb admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:35.689549 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bd4ea2-7b40-4cf5-88a1-c588c076d2fb admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:35.749643 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:24:35.757948 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f8fe37-4aed-46ce-9778-637ca40ec3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:35.758718 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f8fe37-4aed-46ce-9778-637ca40ec3f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:35.759204 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f8fe37-4aed-46ce-9778-637ca40ec3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:35.759692 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f8fe37-4aed-46ce-9778-637ca40ec3f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:35.760583 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f8fe37-4aed-46ce-9778-637ca40ec3f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:35.762771 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f8fe37-4aed-46ce-9778-637ca40ec3f9 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:35.786774 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1433/2862] 10.222.0.13 () {60 vars in 1143 bytes} [Mon Jun 27 21:24:35 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 27 21:24:35.791634 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ec5ba-9d4b-4bb3-b00c-595ebea8ec0c admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:35.811031 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ec5ba-9d4b-4bb3-b00c-595ebea8ec0c admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:35.816675 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c9238d-c015-4ff3-a8ac-764a6911f4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:35.817510 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9238d-c015-4ff3-a8ac-764a6911f4f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:35.818364 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9238d-c015-4ff3-a8ac-764a6911f4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:35.818775 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9238d-c015-4ff3-a8ac-764a6911f4f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:35.819613 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c9238d-c015-4ff3-a8ac-764a6911f4f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:35.821345 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c9238d-c015-4ff3-a8ac-764a6911f4f1 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:35.833417 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1430/2863] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:24:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:24:35.837734 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f8fe37-4aed-46ce-9778-637ca40ec3f9 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:35.848631 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f8fe37-4aed-46ce-9778-637ca40ec3f9 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:35.855051 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0691bb19-a6ae-4a5f-b553-33fc0916de8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:35.855740 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691bb19-a6ae-4a5f-b553-33fc0916de8d admin admin] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:35.856100 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691bb19-a6ae-4a5f-b553-33fc0916de8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:35.856295 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0691bb19-a6ae-4a5f-b553-33fc0916de8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:35.857077 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0691bb19-a6ae-4a5f-b553-33fc0916de8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:35.858726 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0691bb19-a6ae-4a5f-b553-33fc0916de8d admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:35.871991 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1434/2864] 10.222.0.13 () {58 vars in 1121 bytes} [Mon Jun 27 21:24:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 27 21:24:35.876143 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c9238d-c015-4ff3-a8ac-764a6911f4f1 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:35.888533 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c9238d-c015-4ff3-a8ac-764a6911f4f1 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:35.893933 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33a4932-5de6-4c94-a90e-94f681763e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:35.894898 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a4932-5de6-4c94-a90e-94f681763e46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:35.895174 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a4932-5de6-4c94-a90e-94f681763e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:35.895445 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a4932-5de6-4c94-a90e-94f681763e46 admin admin] PATH_INFO: `/v3/projects/30944911cb4b49129983d5c19ca7a6bb/users/c2aa9bdeb9984336bdf103f2d6b1dc13/roles/e4414bde4df44de8aacb36c585a93e07` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:35.913288 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33a4932-5de6-4c94-a90e-94f681763e46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30944911cb4b49129983d5c19ca7a6bb, user_id=c2aa9bdeb9984336bdf103f2d6b1dc13, role_id=e4414bde4df44de8aacb36c585a93e07)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:35.916623 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33a4932-5de6-4c94-a90e-94f681763e46 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:35.932047 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1431/2865] 10.222.0.13 () {60 vars in 1482 bytes} [Mon Jun 27 21:24:35 2022] PUT /identity/v3/projects/30944911cb4b49129983d5c19ca7a6bb/users/c2aa9bdeb9984336bdf103f2d6b1dc13/roles/e4414bde4df44de8aacb36c585a93e07 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:24:35.937657 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ebb84-0507-4383-8f4c-d6ce4b850f54 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:35.938177 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ebb84-0507-4383-8f4c-d6ce4b850f54 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:35.938457 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ebb84-0507-4383-8f4c-d6ce4b850f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:35.958895 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-2c3ebb84-0507-4383-8f4c-d6ce4b850f54 None None] MFA Rules not processed for user `a9e7736617b74291b419604a9ff9d7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:24:35.960410 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-2c3ebb84-0507-4383-8f4c-d6ce4b850f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:36.034201 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1435/2866] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:24:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:24:36.045370 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33a4932-5de6-4c94-a90e-94f681763e46 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:36.047235 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-c33a4932-5de6-4c94-a90e-94f681763e46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:36.110046 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33a4932-5de6-4c94-a90e-94f681763e46 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:36.114362 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080df1d0-697b-4b98-be17-0761cd76a449 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:36.115090 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080df1d0-697b-4b98-be17-0761cd76a449 service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:36.115448 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080df1d0-697b-4b98-be17-0761cd76a449 service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:36.115736 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080df1d0-697b-4b98-be17-0761cd76a449 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:36.126902 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080df1d0-697b-4b98-be17-0761cd76a449 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:36.128362 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080df1d0-697b-4b98-be17-0761cd76a449 service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:36.139188 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1432/2867] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:24:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:24:36.958270 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81471513-6e13-4ce8-a4ab-7648188c1fb3 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:36.959493 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81471513-6e13-4ce8-a4ab-7648188c1fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:36.959493 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81471513-6e13-4ce8-a4ab-7648188c1fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:36.995976 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-81471513-6e13-4ce8-a4ab-7648188c1fb3 None None] MFA Rules not processed for user `c2aa9bdeb9984336bdf103f2d6b1dc13`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:24:36.997415 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-81471513-6e13-4ce8-a4ab-7648188c1fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:37.087642 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1436/2868] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:24:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 27 21:24:37.099595 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080df1d0-697b-4b98-be17-0761cd76a449 service neutron] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:24:37.101674 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-080df1d0-697b-4b98-be17-0761cd76a449 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:37.166556 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080df1d0-697b-4b98-be17-0761cd76a449 service neutron] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:24:37.170177 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f7e9ad-e335-4699-9526-ccece9c429d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '340915e7debe4ce6828bd8a477ed6f16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:24:37.170849 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f7e9ad-e335-4699-9526-ccece9c429d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:37.171130 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f7e9ad-e335-4699-9526-ccece9c429d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:37.171399 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f7e9ad-e335-4699-9526-ccece9c429d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:37.181242 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f7e9ad-e335-4699-9526-ccece9c429d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:24:37.182701 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f7e9ad-e335-4699-9526-ccece9c429d4 service cinder] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:24:37.193384 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1433/2869] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:24:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:24:38.629492 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22dc66-8822-426b-9d1e-7196437dc004 None None] REQUEST_METHOD: `POST` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:38.630157 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22dc66-8822-426b-9d1e-7196437dc004 None None] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:38.630157 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22dc66-8822-426b-9d1e-7196437dc004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:38.649674 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.auth.core [None req-ce22dc66-8822-426b-9d1e-7196437dc004 None None] MFA Rules not processed for user `c2aa9bdeb9984336bdf103f2d6b1dc13`. Rule list: `[]` (Enabled: `True`). {{(pid=80388) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:24:38.650927 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ce22dc66-8822-426b-9d1e-7196437dc004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:38.658304 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1437/2870] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:24:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:24:46.750023 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57937098-e1ea-4582-908a-e607619aa04a None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:24:46.750653 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57937098-e1ea-4582-908a-e607619aa04a None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:24:46.750653 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57937098-e1ea-4582-908a-e607619aa04a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:24:46.769119 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-57937098-e1ea-4582-908a-e607619aa04a None None] MFA Rules not processed for user `e4b46dc825704fbc9cc2505805f22f94`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:24:46.770339 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-57937098-e1ea-4582-908a-e607619aa04a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:24:46.788795 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:24:46.836501 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1434/2871] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:24:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:24:48.643510 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=71,keystone:INSERT=3 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:24:56.827735 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=15 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:25:01.192831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce22dc66-8822-426b-9d1e-7196437dc004 None None] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:25:01.194626 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-ce22dc66-8822-426b-9d1e-7196437dc004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:25:01.208068 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce22dc66-8822-426b-9d1e-7196437dc004 None None] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:25:01.211770 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d971c23-da0e-4c46-a602-325420581469 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:25:01.212425 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d971c23-da0e-4c46-a602-325420581469 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:25:01.212704 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d971c23-da0e-4c46-a602-325420581469 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:25:01.213004 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d971c23-da0e-4c46-a602-325420581469 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:25:01.214831 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-1d971c23-da0e-4c46-a602-325420581469 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:25:01.285654 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d971c23-da0e-4c46-a602-325420581469 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:25:01.287048 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d971c23-da0e-4c46-a602-325420581469 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:25:01.302138 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1438/2872] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:25:01 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:25:11.295431 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=30 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:25:44.917354 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedec947-fdc4-47db-ae4e-5abd562af448 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:25:44.917923 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedec947-fdc4-47db-ae4e-5abd562af448 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:25:44.917923 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedec947-fdc4-47db-ae4e-5abd562af448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:25:44.937603 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-cedec947-fdc4-47db-ae4e-5abd562af448 None None] MFA Rules not processed for user `c2aa9bdeb9984336bdf103f2d6b1dc13`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:25:44.938866 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-cedec947-fdc4-47db-ae4e-5abd562af448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:25:44.947631 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1435/2873] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:25:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:25:50.782247 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d971c23-da0e-4c46-a602-325420581469 service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:25:50.792235 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d971c23-da0e-4c46-a602-325420581469 service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:25:50.798519 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ced43-6b18-45ec-b6da-c9f3e6284943 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:25:50.799440 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ced43-6b18-45ec-b6da-c9f3e6284943 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:25:50.799761 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ced43-6b18-45ec-b6da-c9f3e6284943 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:25:50.800093 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ced43-6b18-45ec-b6da-c9f3e6284943 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:25:50.803010 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-400ced43-6b18-45ec-b6da-c9f3e6284943 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:25:50.887555 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400ced43-6b18-45ec-b6da-c9f3e6284943 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:25:50.889002 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400ced43-6b18-45ec-b6da-c9f3e6284943 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:25:50.901285 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1439/2874] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:25:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:25:54.931193 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=3 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:25:56.063764 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedec947-fdc4-47db-ae4e-5abd562af448 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:25:56.075057 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedec947-fdc4-47db-ae4e-5abd562af448 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:25:56.079438 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a61f2c5-042f-4f4f-9636-68a75d500228 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:25:56.080091 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a61f2c5-042f-4f4f-9636-68a75d500228 service neutron] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:25:56.080367 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a61f2c5-042f-4f4f-9636-68a75d500228 service neutron] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:25:56.080641 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a61f2c5-042f-4f4f-9636-68a75d500228 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:25:56.082666 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-6a61f2c5-042f-4f4f-9636-68a75d500228 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:25:56.097564 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a61f2c5-042f-4f4f-9636-68a75d500228 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:25:56.099000 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a61f2c5-042f-4f4f-9636-68a75d500228 service neutron] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:25:56.110552 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1436/2875] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:25:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:26:00.895805 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=34 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:26:06.105801 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=6 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:26:32.353110 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ced43-6b18-45ec-b6da-c9f3e6284943 service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:26:32.364431 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ced43-6b18-45ec-b6da-c9f3e6284943 service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:26:32.372848 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c1ee2f-f546-4513-bead-11d87f111732 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:26:32.373755 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1ee2f-f546-4513-bead-11d87f111732 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:26:32.374036 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1ee2f-f546-4513-bead-11d87f111732 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:26:32.374490 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1ee2f-f546-4513-bead-11d87f111732 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:26:32.377972 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-13c1ee2f-f546-4513-bead-11d87f111732 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:26:32.394243 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c1ee2f-f546-4513-bead-11d87f111732 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:26:32.396482 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c1ee2f-f546-4513-bead-11d87f111732 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:26:32.411835 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1440/2876] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:26:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:26:41.937323 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fcab87-7e21-4983-814c-c2b75c2444d0 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:26:41.937765 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fcab87-7e21-4983-814c-c2b75c2444d0 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:26:41.937962 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fcab87-7e21-4983-814c-c2b75c2444d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:26:41.957557 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-e6fcab87-7e21-4983-814c-c2b75c2444d0 None None] MFA Rules not processed for user `f67110d591a94537a3df6a2d835370f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:26:41.958979 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-e6fcab87-7e21-4983-814c-c2b75c2444d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:26:42.015105 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1437/2877] 10.222.0.13 () {58 vars in 967 bytes} [Mon Jun 27 21:26:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:26:42.058015 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c1ee2f-f546-4513-bead-11d87f111732 service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:26:42.059527 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-13c1ee2f-f546-4513-bead-11d87f111732 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:26:42.144102 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c1ee2f-f546-4513-bead-11d87f111732 service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:26:42.150344 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d3d169-fd45-4e9a-a0b0-d18331f97ae4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bed6b231ee647d8a350da055d8e59d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:26:42.150993 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d3d169-fd45-4e9a-a0b0-d18331f97ae4 service swift] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:26:42.151275 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d3d169-fd45-4e9a-a0b0-d18331f97ae4 service swift] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:26:42.151551 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d3d169-fd45-4e9a-a0b0-d18331f97ae4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:26:42.160750 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d3d169-fd45-4e9a-a0b0-d18331f97ae4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:26:42.162541 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d3d169-fd45-4e9a-a0b0-d18331f97ae4 service swift] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:26:42.171348 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1441/2878] 10.222.0.13 () {60 vars in 1384 bytes} [Mon Jun 27 21:26:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 27 21:26:43.014332 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fcab87-7e21-4983-814c-c2b75c2444d0 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:26:43.032704 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fcab87-7e21-4983-814c-c2b75c2444d0 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:26:43.038152 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c83eb87-efd3-4ede-8eef-465fe946a7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:26:43.039133 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83eb87-efd3-4ede-8eef-465fe946a7a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:26:43.039484 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83eb87-efd3-4ede-8eef-465fe946a7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:26:43.039826 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c83eb87-efd3-4ede-8eef-465fe946a7a5 admin admin] PATH_INFO: `/v3/users/c2aa9bdeb9984336bdf103f2d6b1dc13` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:26:43.041355 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c83eb87-efd3-4ede-8eef-465fe946a7a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2aa9bdeb9984336bdf103f2d6b1dc13)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:26:43.043393 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c83eb87-efd3-4ede-8eef-465fe946a7a5 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:26:43.096823 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-3c83eb87-efd3-4ede-8eef-465fe946a7a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2aa9bdeb9984336bdf103f2d6b1dc13'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:26:43.104768 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-3c83eb87-efd3-4ede-8eef-465fe946a7a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2aa9bdeb9984336bdf103f2d6b1dc13'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:26:43.105524 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-3c83eb87-efd3-4ede-8eef-465fe946a7a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2aa9bdeb9984336bdf103f2d6b1dc13'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:26:43.106149 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-3c83eb87-efd3-4ede-8eef-465fe946a7a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2aa9bdeb9984336bdf103f2d6b1dc13'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:26:43.114288 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-3c83eb87-efd3-4ede-8eef-465fe946a7a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2aa9bdeb9984336bdf103f2d6b1dc13'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:26:43.134065 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1438/2879] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:26:43 2022] DELETE /identity/v3/users/c2aa9bdeb9984336bdf103f2d6b1dc13 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:26:43.313322 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d3d169-fd45-4e9a-a0b0-d18331f97ae4 service swift] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:26:43.324938 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d3d169-fd45-4e9a-a0b0-d18331f97ae4 service swift] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:26:43.396713 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f135f-1d4a-4a5e-bc51-31c535434886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:26:43.397626 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f135f-1d4a-4a5e-bc51-31c535434886 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:26:43.397905 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f135f-1d4a-4a5e-bc51-31c535434886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:26:43.398208 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f135f-1d4a-4a5e-bc51-31c535434886 admin admin] PATH_INFO: `/v3/projects/30944911cb4b49129983d5c19ca7a6bb` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:26:43.399451 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729f135f-1d4a-4a5e-bc51-31c535434886 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30944911cb4b49129983d5c19ca7a6bb)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:26:43.401414 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729f135f-1d4a-4a5e-bc51-31c535434886 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:26:43.452222 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-729f135f-1d4a-4a5e-bc51-31c535434886 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30944911cb4b49129983d5c19ca7a6bb', 'request_id': 'req-729f135f-1d4a-4a5e-bc51-31c535434886'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:26:43.460636 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-729f135f-1d4a-4a5e-bc51-31c535434886 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30944911cb4b49129983d5c19ca7a6bb', 'request_id': 'req-729f135f-1d4a-4a5e-bc51-31c535434886'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:26:43.472696 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-729f135f-1d4a-4a5e-bc51-31c535434886 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30944911cb4b49129983d5c19ca7a6bb', 'request_id': 'req-729f135f-1d4a-4a5e-bc51-31c535434886'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:26:43.488219 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-729f135f-1d4a-4a5e-bc51-31c535434886 admin admin] The token cache is being invalidate because project 30944911cb4b49129983d5c19ca7a6bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:26:43.488498 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-729f135f-1d4a-4a5e-bc51-31c535434886 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:26:43.489249 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-729f135f-1d4a-4a5e-bc51-31c535434886 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:26:43.491166 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1442/2880] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:26:43 2022] DELETE /identity/v3/projects/30944911cb4b49129983d5c19ca7a6bb => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:26:44.774267 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214d33c-ebd6-457f-81d8-34d31876d989 None None] REQUEST_METHOD: `POST` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:26:44.774843 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214d33c-ebd6-457f-81d8-34d31876d989 None None] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:26:44.774843 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214d33c-ebd6-457f-81d8-34d31876d989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:26:44.793805 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.auth.core [None req-c214d33c-ebd6-457f-81d8-34d31876d989 None None] MFA Rules not processed for user `e4b46dc825704fbc9cc2505805f22f94`. Rule list: `[]` (Enabled: `True`). {{(pid=80389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 27 21:26:44.795249 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-c214d33c-ebd6-457f-81d8-34d31876d989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:26:44.861192 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1439/2881] 10.222.0.13 () {58 vars in 957 bytes} [Mon Jun 27 21:26:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 27 21:26:45.784985 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f135f-1d4a-4a5e-bc51-31c535434886 admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:26:45.786572 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-729f135f-1d4a-4a5e-bc51-31c535434886 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:26:45.805469 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:26:45.909238 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f135f-1d4a-4a5e-bc51-31c535434886 admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:26:45.915570 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d877aa9-f73d-4a2d-86f2-5659b7d6640b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0437c45d72cd4a38b9d48ee6f50946fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:26:45.916265 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d877aa9-f73d-4a2d-86f2-5659b7d6640b service placement] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:26:45.916553 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d877aa9-f73d-4a2d-86f2-5659b7d6640b service placement] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:26:45.917608 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d877aa9-f73d-4a2d-86f2-5659b7d6640b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:26:45.919536 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-1d877aa9-f73d-4a2d-86f2-5659b7d6640b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:26:45.933280 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d877aa9-f73d-4a2d-86f2-5659b7d6640b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:26:45.934591 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d877aa9-f73d-4a2d-86f2-5659b7d6640b service placement] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:26:45.946121 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1443/2882] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:26:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:26:50.117386 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214d33c-ebd6-457f-81d8-34d31876d989 None None] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:26:50.130590 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214d33c-ebd6-457f-81d8-34d31876d989 None None] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:26:50.135630 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e418ad6c-492b-441d-91d7-60d5fc56e91f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0437c45d72cd4a38b9d48ee6f50946fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:26:50.136291 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e418ad6c-492b-441d-91d7-60d5fc56e91f service placement] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:26:50.136577 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e418ad6c-492b-441d-91d7-60d5fc56e91f service placement] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:26:50.136852 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e418ad6c-492b-441d-91d7-60d5fc56e91f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:26:50.138675 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-e418ad6c-492b-441d-91d7-60d5fc56e91f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:26:50.151071 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e418ad6c-492b-441d-91d7-60d5fc56e91f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:26:50.152597 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e418ad6c-492b-441d-91d7-60d5fc56e91f service placement] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:26:50.163660 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1440/2883] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:26:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:26:50.175036 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d877aa9-f73d-4a2d-86f2-5659b7d6640b service placement] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:26:50.177397 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-1d877aa9-f73d-4a2d-86f2-5659b7d6640b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:26:50.250144 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d877aa9-f73d-4a2d-86f2-5659b7d6640b service placement] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:26:50.255547 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eec55c-ba6d-485a-bc9d-e932b45c8c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:26:50.256243 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eec55c-ba6d-485a-bc9d-e932b45c8c8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:26:50.256525 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eec55c-ba6d-485a-bc9d-e932b45c8c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:26:50.256803 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eec55c-ba6d-485a-bc9d-e932b45c8c8b admin admin] PATH_INFO: `/v3/users/4d867759b4f249a393598964774a22fa` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:26:50.258116 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eec55c-ba6d-485a-bc9d-e932b45c8c8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d867759b4f249a393598964774a22fa)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:26:50.260087 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eec55c-ba6d-485a-bc9d-e932b45c8c8b admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:26:50.314095 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-86eec55c-ba6d-485a-bc9d-e932b45c8c8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d867759b4f249a393598964774a22fa'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:26:50.321851 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-86eec55c-ba6d-485a-bc9d-e932b45c8c8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d867759b4f249a393598964774a22fa'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:26:50.322529 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-86eec55c-ba6d-485a-bc9d-e932b45c8c8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d867759b4f249a393598964774a22fa'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:26:50.323145 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-86eec55c-ba6d-485a-bc9d-e932b45c8c8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d867759b4f249a393598964774a22fa'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:26:50.333352 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-86eec55c-ba6d-485a-bc9d-e932b45c8c8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d867759b4f249a393598964774a22fa'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:26:50.351463 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1444/2884] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:26:50 2022] DELETE /identity/v3/users/4d867759b4f249a393598964774a22fa => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:26:50.534131 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e418ad6c-492b-441d-91d7-60d5fc56e91f service placement] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:26:50.547752 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e418ad6c-492b-441d-91d7-60d5fc56e91f service placement] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:26:50.553206 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:26:50.624581 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74925bf1-9156-44c8-b20a-e87df4f72926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:26:50.625500 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74925bf1-9156-44c8-b20a-e87df4f72926 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:26:50.628299 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74925bf1-9156-44c8-b20a-e87df4f72926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:26:50.628621 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74925bf1-9156-44c8-b20a-e87df4f72926 admin admin] PATH_INFO: `/v3/projects/c9f86d6febe04e8f9745d787914b6d5c` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:26:50.629964 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74925bf1-9156-44c8-b20a-e87df4f72926 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9f86d6febe04e8f9745d787914b6d5c)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:26:50.631957 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74925bf1-9156-44c8-b20a-e87df4f72926 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:26:50.691378 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-74925bf1-9156-44c8-b20a-e87df4f72926 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9f86d6febe04e8f9745d787914b6d5c', 'request_id': 'req-74925bf1-9156-44c8-b20a-e87df4f72926'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:26:50.704637 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-74925bf1-9156-44c8-b20a-e87df4f72926 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9f86d6febe04e8f9745d787914b6d5c', 'request_id': 'req-74925bf1-9156-44c8-b20a-e87df4f72926'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:26:50.717982 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-74925bf1-9156-44c8-b20a-e87df4f72926 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9f86d6febe04e8f9745d787914b6d5c', 'request_id': 'req-74925bf1-9156-44c8-b20a-e87df4f72926'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:26:50.731393 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-74925bf1-9156-44c8-b20a-e87df4f72926 admin admin] The token cache is being invalidate because project c9f86d6febe04e8f9745d787914b6d5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:26:50.732709 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-74925bf1-9156-44c8-b20a-e87df4f72926 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:26:50.733748 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-74925bf1-9156-44c8-b20a-e87df4f72926 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:26:50.736117 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1441/2885] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:26:50 2022] DELETE /identity/v3/projects/c9f86d6febe04e8f9745d787914b6d5c => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:26:50.740229 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eec55c-ba6d-485a-bc9d-e932b45c8c8b admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:26:50.741747 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-86eec55c-ba6d-485a-bc9d-e932b45c8c8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:26:50.755334 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eec55c-ba6d-485a-bc9d-e932b45c8c8b admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:26:50.769385 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:26:50.822363 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a19b4-16f3-4fa0-8fb7-e4198168940c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:26:50.823293 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a19b4-16f3-4fa0-8fb7-e4198168940c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:26:50.823551 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a19b4-16f3-4fa0-8fb7-e4198168940c admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:26:50.823820 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a19b4-16f3-4fa0-8fb7-e4198168940c admin admin] PATH_INFO: `/v3/users/a7d3c0ed840a4d559e1f0e227de8f024` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:26:50.825227 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16a19b4-16f3-4fa0-8fb7-e4198168940c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7d3c0ed840a4d559e1f0e227de8f024)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:26:50.827055 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16a19b4-16f3-4fa0-8fb7-e4198168940c admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:26:50.882549 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b16a19b4-16f3-4fa0-8fb7-e4198168940c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7d3c0ed840a4d559e1f0e227de8f024'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:26:50.890065 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b16a19b4-16f3-4fa0-8fb7-e4198168940c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7d3c0ed840a4d559e1f0e227de8f024'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:26:50.890812 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b16a19b4-16f3-4fa0-8fb7-e4198168940c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7d3c0ed840a4d559e1f0e227de8f024'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:26:50.891648 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b16a19b4-16f3-4fa0-8fb7-e4198168940c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7d3c0ed840a4d559e1f0e227de8f024'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:26:50.901373 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-b16a19b4-16f3-4fa0-8fb7-e4198168940c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7d3c0ed840a4d559e1f0e227de8f024'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:26:50.920730 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1445/2886] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:26:50 2022] DELETE /identity/v3/users/a7d3c0ed840a4d559e1f0e227de8f024 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:26:51.119053 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74925bf1-9156-44c8-b20a-e87df4f72926 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:26:51.122394 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-74925bf1-9156-44c8-b20a-e87df4f72926 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:26:51.148390 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74925bf1-9156-44c8-b20a-e87df4f72926 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:26:51.215145 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3454021-4618-4d1d-815d-ae240b547706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:26:51.216395 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3454021-4618-4d1d-815d-ae240b547706 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:26:51.216807 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3454021-4618-4d1d-815d-ae240b547706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:26:51.217273 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3454021-4618-4d1d-815d-ae240b547706 admin admin] PATH_INFO: `/v3/projects/1bc2225ec19243f9b09cc1847ef9bce7` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:26:51.219049 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3454021-4618-4d1d-815d-ae240b547706 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bc2225ec19243f9b09cc1847ef9bce7)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:26:51.222190 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3454021-4618-4d1d-815d-ae240b547706 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:26:51.275942 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:26:51.283294 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-f3454021-4618-4d1d-815d-ae240b547706 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bc2225ec19243f9b09cc1847ef9bce7', 'request_id': 'req-f3454021-4618-4d1d-815d-ae240b547706'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:26:51.293692 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-f3454021-4618-4d1d-815d-ae240b547706 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bc2225ec19243f9b09cc1847ef9bce7', 'request_id': 'req-f3454021-4618-4d1d-815d-ae240b547706'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:26:51.304324 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-f3454021-4618-4d1d-815d-ae240b547706 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bc2225ec19243f9b09cc1847ef9bce7', 'request_id': 'req-f3454021-4618-4d1d-815d-ae240b547706'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:26:51.314979 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-f3454021-4618-4d1d-815d-ae240b547706 admin admin] The token cache is being invalidate because project 1bc2225ec19243f9b09cc1847ef9bce7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:26:51.315293 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-f3454021-4618-4d1d-815d-ae240b547706 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:26:51.315942 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-f3454021-4618-4d1d-815d-ae240b547706 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:26:51.317591 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1442/2887] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:26:51 2022] DELETE /identity/v3/projects/1bc2225ec19243f9b09cc1847ef9bce7 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:26:54.218404 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a19b4-16f3-4fa0-8fb7-e4198168940c admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:26:54.220278 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-b16a19b4-16f3-4fa0-8fb7-e4198168940c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:26:54.286020 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a19b4-16f3-4fa0-8fb7-e4198168940c admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:26:54.341304 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e5951-7edf-4d08-b811-ace2a3fd233e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:26:54.341959 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e5951-7edf-4d08-b811-ace2a3fd233e service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:26:54.342236 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e5951-7edf-4d08-b811-ace2a3fd233e service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:26:54.342503 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e5951-7edf-4d08-b811-ace2a3fd233e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:26:54.344866 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-2b8e5951-7edf-4d08-b811-ace2a3fd233e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:26:54.351347 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:26:54.422630 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8e5951-7edf-4d08-b811-ace2a3fd233e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:26:54.425229 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8e5951-7edf-4d08-b811-ace2a3fd233e service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:26:54.441495 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1446/2888] 10.222.0.13 () {60 vars in 1396 bytes} [Mon Jun 27 21:26:54 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3246 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:26:59.050096 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3454021-4618-4d1d-815d-ae240b547706 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:26:59.052543 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-f3454021-4618-4d1d-815d-ae240b547706 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:26:59.146821 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3454021-4618-4d1d-815d-ae240b547706 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:26:59.161601 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34e30a4-73ff-4f68-8ab0-8e4d1db7b915 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:26:59.162276 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e30a4-73ff-4f68-8ab0-8e4d1db7b915 service nova] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:26:59.162559 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e30a4-73ff-4f68-8ab0-8e4d1db7b915 service nova] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:26:59.162835 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e30a4-73ff-4f68-8ab0-8e4d1db7b915 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:26:59.165113 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-a34e30a4-73ff-4f68-8ab0-8e4d1db7b915 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:26:59.178793 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34e30a4-73ff-4f68-8ab0-8e4d1db7b915 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:26:59.180645 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34e30a4-73ff-4f68-8ab0-8e4d1db7b915 service nova] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:26:59.192785 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1443/2889] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:26:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:27:04.434462 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=25 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:27:09.187109 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=43,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:28:32.249232 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e5951-7edf-4d08-b811-ace2a3fd233e service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:32.251757 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-2b8e5951-7edf-4d08-b811-ace2a3fd233e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:32.281653 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e5951-7edf-4d08-b811-ace2a3fd233e service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:32.295370 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477789a-15e4-435c-80ff-b16104bf80ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:32.296665 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477789a-15e4-435c-80ff-b16104bf80ee service nova] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:32.297301 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477789a-15e4-435c-80ff-b16104bf80ee service nova] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:32.297630 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f477789a-15e4-435c-80ff-b16104bf80ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:32.300917 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-f477789a-15e4-435c-80ff-b16104bf80ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:32.333055 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f477789a-15e4-435c-80ff-b16104bf80ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:32.334928 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f477789a-15e4-435c-80ff-b16104bf80ee service nova] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:32.353697 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1447/2890] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:28:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:28:34.890153 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34e30a4-73ff-4f68-8ab0-8e4d1db7b915 service nova] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:34.960607 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34e30a4-73ff-4f68-8ab0-8e4d1db7b915 service nova] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:34.970764 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a882c1f0-6f88-416a-a2bb-4c273c9c7a17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1366958034d540b4ba2feb421a6afabc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:34.972142 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a882c1f0-6f88-416a-a2bb-4c273c9c7a17 service nova] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:34.973014 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a882c1f0-6f88-416a-a2bb-4c273c9c7a17 service nova] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:34.973698 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a882c1f0-6f88-416a-a2bb-4c273c9c7a17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:34.977427 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-a882c1f0-6f88-416a-a2bb-4c273c9c7a17 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:35.037514 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a882c1f0-6f88-416a-a2bb-4c273c9c7a17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:35.040165 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a882c1f0-6f88-416a-a2bb-4c273c9c7a17 service nova] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:35.054446 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1444/2891] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:28:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:28:42.344169 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=6 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:28:45.047961 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=28 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:28:45.572977 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477789a-15e4-435c-80ff-b16104bf80ee service nova] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:45.574730 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-f477789a-15e4-435c-80ff-b16104bf80ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:45.691270 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f477789a-15e4-435c-80ff-b16104bf80ee service nova] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:45.699287 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd1d2db-f1cd-4729-be8c-5072d2a117f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4e105858324391b5142d57e34aff9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:45.701732 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd1d2db-f1cd-4729-be8c-5072d2a117f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:45.702313 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd1d2db-f1cd-4729-be8c-5072d2a117f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:45.702810 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd1d2db-f1cd-4729-be8c-5072d2a117f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:45.707779 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-8dd1d2db-f1cd-4729-be8c-5072d2a117f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:45.836108 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd1d2db-f1cd-4729-be8c-5072d2a117f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:45.837805 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd1d2db-f1cd-4729-be8c-5072d2a117f6 service neutron] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:45.866726 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1448/2892] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:28:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 27 21:28:54.471550 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a882c1f0-6f88-416a-a2bb-4c273c9c7a17 service nova] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:54.474161 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-a882c1f0-6f88-416a-a2bb-4c273c9c7a17 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:54.494762 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a882c1f0-6f88-416a-a2bb-4c273c9c7a17 service nova] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:54.501357 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fece662-58a0-4a19-aead-7ea76546e9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:54.502286 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fece662-58a0-4a19-aead-7ea76546e9e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:54.502570 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fece662-58a0-4a19-aead-7ea76546e9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:54.502846 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fece662-58a0-4a19-aead-7ea76546e9e3 admin admin] PATH_INFO: `/v3/users/e4b46dc825704fbc9cc2505805f22f94` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:54.504405 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fece662-58a0-4a19-aead-7ea76546e9e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4b46dc825704fbc9cc2505805f22f94)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:54.506450 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fece662-58a0-4a19-aead-7ea76546e9e3 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:54.572441 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-7fece662-58a0-4a19-aead-7ea76546e9e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4b46dc825704fbc9cc2505805f22f94'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:54.581892 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-7fece662-58a0-4a19-aead-7ea76546e9e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4b46dc825704fbc9cc2505805f22f94'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:54.582701 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-7fece662-58a0-4a19-aead-7ea76546e9e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4b46dc825704fbc9cc2505805f22f94'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:54.583435 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-7fece662-58a0-4a19-aead-7ea76546e9e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4b46dc825704fbc9cc2505805f22f94'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:54.596365 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-7fece662-58a0-4a19-aead-7ea76546e9e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4b46dc825704fbc9cc2505805f22f94'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:54.631365 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1445/2893] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:28:54 2022] DELETE /identity/v3/users/e4b46dc825704fbc9cc2505805f22f94 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:28:54.864040 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd1d2db-f1cd-4729-be8c-5072d2a117f6 service neutron] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:54.876801 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd1d2db-f1cd-4729-be8c-5072d2a117f6 service neutron] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:54.967461 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a0d62-3a95-4f68-8658-a8b248b08dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:54.968347 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a0d62-3a95-4f68-8658-a8b248b08dcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:54.968654 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a0d62-3a95-4f68-8658-a8b248b08dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:54.968956 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a0d62-3a95-4f68-8658-a8b248b08dcd admin admin] PATH_INFO: `/v3/projects/824cb65198bf417fbc4baeae349d358d` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:54.970380 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34a0d62-3a95-4f68-8658-a8b248b08dcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=824cb65198bf417fbc4baeae349d358d)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:54.972375 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34a0d62-3a95-4f68-8658-a8b248b08dcd admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:55.000101 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=100 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:28:55.037895 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-d34a0d62-3a95-4f68-8658-a8b248b08dcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '824cb65198bf417fbc4baeae349d358d', 'request_id': 'req-d34a0d62-3a95-4f68-8658-a8b248b08dcd'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:55.046089 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-d34a0d62-3a95-4f68-8658-a8b248b08dcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '824cb65198bf417fbc4baeae349d358d', 'request_id': 'req-d34a0d62-3a95-4f68-8658-a8b248b08dcd'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:55.056660 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-d34a0d62-3a95-4f68-8658-a8b248b08dcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '824cb65198bf417fbc4baeae349d358d', 'request_id': 'req-d34a0d62-3a95-4f68-8658-a8b248b08dcd'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:55.070433 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-d34a0d62-3a95-4f68-8658-a8b248b08dcd admin admin] The token cache is being invalidate because project 824cb65198bf417fbc4baeae349d358d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:28:55.070860 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-d34a0d62-3a95-4f68-8658-a8b248b08dcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:55.071631 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-d34a0d62-3a95-4f68-8658-a8b248b08dcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:55.073587 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1449/2894] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:28:54 2022] DELETE /identity/v3/projects/824cb65198bf417fbc4baeae349d358d => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:28:55.079502 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fece662-58a0-4a19-aead-7ea76546e9e3 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:55.081874 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-7fece662-58a0-4a19-aead-7ea76546e9e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:55.105797 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fece662-58a0-4a19-aead-7ea76546e9e3 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:55.180116 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff2a03-1e54-41c3-a196-e10f85d23458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:55.181161 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff2a03-1e54-41c3-a196-e10f85d23458 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:55.181560 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff2a03-1e54-41c3-a196-e10f85d23458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:55.181883 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff2a03-1e54-41c3-a196-e10f85d23458 admin admin] PATH_INFO: `/v3/users/4640e362bf7349deb51ab58a376091e5` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:55.183348 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff2a03-1e54-41c3-a196-e10f85d23458 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4640e362bf7349deb51ab58a376091e5)` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:55.185658 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ff2a03-1e54-41c3-a196-e10f85d23458 admin admin] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:55.251289 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-26ff2a03-1e54-41c3-a196-e10f85d23458 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4640e362bf7349deb51ab58a376091e5'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:55.265211 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-26ff2a03-1e54-41c3-a196-e10f85d23458 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4640e362bf7349deb51ab58a376091e5'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:55.266073 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-26ff2a03-1e54-41c3-a196-e10f85d23458 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4640e362bf7349deb51ab58a376091e5'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:55.266761 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-26ff2a03-1e54-41c3-a196-e10f85d23458 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4640e362bf7349deb51ab58a376091e5'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:55.277388 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.notifications [None req-26ff2a03-1e54-41c3-a196-e10f85d23458 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4640e362bf7349deb51ab58a376091e5'} {{(pid=80389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:55.287627 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:28:55.311210 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1446/2895] 10.222.0.13 () {58 vars in 1223 bytes} [Mon Jun 27 21:28:55 2022] DELETE /identity/v3/users/4640e362bf7349deb51ab58a376091e5 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 27 21:28:55.579071 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a0d62-3a95-4f68-8658-a8b248b08dcd admin admin] Authenticating user token {{(pid=80388) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:28:55.581135 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.fernet_utils [None req-d34a0d62-3a95-4f68-8658-a8b248b08dcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80388) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:28:55.609307 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a0d62-3a95-4f68-8658-a8b248b08dcd admin admin] Validating token access rules against request {{(pid=80388) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:28:55.702603 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8cd6db-873a-45f9-a83f-41cc71d2a690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9e7736617b74291b419604a9ff9d7e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b6e5cf8b46f34e30a9b900f3cd1507c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80388) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:28:55.703618 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cd6db-873a-45f9-a83f-41cc71d2a690 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:28:55.704043 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cd6db-873a-45f9-a83f-41cc71d2a690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:28:55.704485 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cd6db-873a-45f9-a83f-41cc71d2a690 admin admin] PATH_INFO: `/v3/projects/3a7fee95fa2a436bbe9c4ca355a9a1a6` {{(pid=80388) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:28:55.706137 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8cd6db-873a-45f9-a83f-41cc71d2a690 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a7fee95fa2a436bbe9c4ca355a9a1a6)` {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:28:55.708361 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8cd6db-873a-45f9-a83f-41cc71d2a690 admin admin] RBAC: Authorization granted {{(pid=80388) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:28:55.770816 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8c8cd6db-873a-45f9-a83f-41cc71d2a690 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a7fee95fa2a436bbe9c4ca355a9a1a6', 'request_id': 'req-8c8cd6db-873a-45f9-a83f-41cc71d2a690'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:55.779582 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8c8cd6db-873a-45f9-a83f-41cc71d2a690 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a7fee95fa2a436bbe9c4ca355a9a1a6', 'request_id': 'req-8c8cd6db-873a-45f9-a83f-41cc71d2a690'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:55.790173 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8c8cd6db-873a-45f9-a83f-41cc71d2a690 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a7fee95fa2a436bbe9c4ca355a9a1a6', 'request_id': 'req-8c8cd6db-873a-45f9-a83f-41cc71d2a690'} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:55.804528 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8c8cd6db-873a-45f9-a83f-41cc71d2a690 admin admin] The token cache is being invalidate because project 3a7fee95fa2a436bbe9c4ca355a9a1a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80388) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 27 21:28:55.804862 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8c8cd6db-873a-45f9-a83f-41cc71d2a690 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:55.805598 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG keystone.notifications [None req-8c8cd6db-873a-45f9-a83f-41cc71d2a690 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80388) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 27 21:28:55.807310 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: [pid: 80388|app: 0|req: 1450/2896] 10.222.0.13 () {58 vars in 1232 bytes} [Mon Jun 27 21:28:55 2022] DELETE /identity/v3/projects/3a7fee95fa2a436bbe9c4ca355a9a1a6 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 27 21:29:05.318658 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:29:05.801154 dv-cinder-847732-1-iscsi devstack@keystone.service[80388]: DEBUG dbcounter [-] [80388] Writing DB stats keystone:SELECT=67,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80388) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 27 21:29:52.605167 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff2a03-1e54-41c3-a196-e10f85d23458 admin admin] Authenticating user token {{(pid=80389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 27 21:29:52.607939 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-26ff2a03-1e54-41c3-a196-e10f85d23458 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:29:52.717838 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff2a03-1e54-41c3-a196-e10f85d23458 admin admin] Validating token access rules against request {{(pid=80389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 27 21:29:52.807782 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0a742-d13a-4173-a467-7ee4e8efdde5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0437c45d72cd4a38b9d48ee6f50946fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '334438c929c44e489e9a3b93e6d37231', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 27 21:29:52.809435 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0a742-d13a-4173-a467-7ee4e8efdde5 service placement] REQUEST_METHOD: `GET` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 27 21:29:52.809660 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0a742-d13a-4173-a467-7ee4e8efdde5 service placement] SCRIPT_NAME: `/identity` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 27 21:29:52.810161 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0a742-d13a-4173-a467-7ee4e8efdde5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 27 21:29:52.813476 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.fernet_utils [None req-c4b0a742-d13a-4173-a467-7ee4e8efdde5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 27 21:29:52.835391 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b0a742-d13a-4173-a467-7ee4e8efdde5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 27 21:29:52.837077 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b0a742-d13a-4173-a467-7ee4e8efdde5 service placement] RBAC: Authorization granted {{(pid=80389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 27 21:29:52.849850 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: [pid: 80389|app: 0|req: 1447/2897] 10.222.0.13 () {60 vars in 1365 bytes} [Mon Jun 27 21:29:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 27 21:30:02.844307 dv-cinder-847732-1-iscsi devstack@keystone.service[80389]: DEBUG dbcounter [-] [80389] Writing DB stats keystone:SELECT=67 {{(pid=80389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}